Configuration

Hardware

Login

While the hardware provided for Lighthouse may vary by PI, the login nodes and process remains uniform.  All Lighthouse users should login to lighthouse.arc-ts.umich.edu in order to submit jobs.

Networking

The compute nodes are all interconnected with InfiniBand networking. In addition to the InfiniBand, there is a gigabit Ethernet network that also connects all of the nodes.

Storage

Storage on Lighthouse: /home and /scratch

On the Lighthouse cluster, users have access to two main types of storage: /home and /scratch. Each serves a different purpose:

  • /home is for personal, non-project work like scripts, small data files, and software builds.
  • /scratch is for active project data needed while running jobs. It’s larger but temporary.

Both directories have specific rules about who can access your files.

File Permissions:

To protect your data and the system, files or folders that are set to be readable or writable by “other” (users outside your account or group) will have those permissions automatically removed. Always set appropriate permissions to limit access to just yourself or your project group. If you need a group to manage access to a directory or files, let us know.

Home Directories

Every user is assigned a /home directory for non-project work. This space is intended for storing scripts, small data files, software builds, and other personal files unrelated to specific funded projects.

  • Each /home directory has a hard quota of 80 GB.
  • Files stored here should only be accessible by the user and their primary group(s).
  • Files or directories with global (other) read or write permissions will have those permissions automatically removed to maintain system security and data integrity.

Use this space for development, light workflows, and other general-purpose computing needs. For large datasets or collaborative work, use the /scratch directory structure described below.

Scratch Directories

Every user has a /scratch folder for each Slurm account they belong to. This space is designed for fast access to active data and should only be used for files needed to run current jobs on the cluster.

  • /scratch uses the Turbo high-performance file system.
  • It is not backed up, so do not store important or long-term files here. Move important data to a durable location when your jobs are done.
  • Each Slurm account also has a shared_data folder where all users in the account can share files.
  • The folders are set up so that files can be accessed by anyone in the same Slurm account group, to make working together easier.

 

Example:
/scratch/myproject_root/myproject1
/scratch/myproject_root/myproject1/bob
/scratch/myproject_root/myproject1/shared_data

Slurm Accounts and Funding:

Each Slurm account may match a different source of funding:

  • myproject0 → UMRCP (free allocation)
  • myproject1 → Research Allocation X (paid)
  • myproject2 → Research Allocation Y (paid)

 

Make sure to use the correct account for each project so your usage is tracked properly.

See our list of storage options to find the best fit for your needs.

Operation

Computing jobs on Lighthouse are managed completely through Slurm. See the Lighthouse User Guide for directions on how to submit and manage jobs.

Software

There are three layers of software on Lighthouse:

Operating Software

The Lighthouse cluster runs Redhat 8. We update the operating system on Lighthouse as Redhat releases new versions and our library of third-party applications offers support. Due to the need to support several types of drivers (AFS and Lustre file system drivers, InfiniBand network drivers and NVIDIA GPU drivers) and dozens of third party applications, we are cautious in upgrading and can lag Redhat releases by months.

Compilers, Parallel, and Scientific Libraries

Lighthouse supports the Gnu Compiler Collection, the Intel Compilers, and the PGI Compilers for C and Fortran. The Lighthouse cluster’s parallel library is OpenMPI, and the default versions are 1.10.7 (i686) and 3.1.2 (x86_64), and there are limited earlier versions available.  Lighthouse provides the Intel Math Kernel Library (MKL) set of high-performance mathematical libraries. Other common scientific libraries are compiled from source and include HDF5, NetCDF, FFTW3, Boost, and others.