By default, Open OnDemand will start you in your home directory. That has a path of /home/$USER where $USER is your uniqname. Note that your home directory is a local storage location mounted on the cluster and is limited to 80GB. You may want to move your program files to this space if want to run jobs from this location. However, note the small size of this space and plan accordingly for where you want to call your program and files from this location or alternatively you can run natively on Turbo.
Some users have access to /scratch on the clusters and scratch has limits. Work with your Primary Investigator to see if /scratch is needed and if you have access.
Many users have a Turbo volume they can access to run programs against their data in this file path. The file path to your turbo is /nfs/turbo/<volume name> where volume name is your Turbo volume name as assigned or your shared Turbo location of data files if you are collaboratively working on a research project.
Be sure to have the data and code files you need in your preferred directory. You may want to use these methods to transfer files:
- Turbo on mounted on your machine to transfer files where you can copy files to and from Turbo
- SFTP to move files to and from the cluster (your home directory, scratch or your Turbo volume)
- Globus File Transfer Tool
Get your files moved to the cluster to use
You can use SFTP (best for simple transfers of small files) or Globus (best for large files or a commonly used endpoint) to transfer data to your /home directory.
SFTP: Mac or Windows using FileZilla
- Open FileZilla and click the “Site Manager” button
- Create a New Site, which you can name “Great Lakes” or something similar
- Select the “SFTP (SSH File Transfer Protocol)” option
- In the Host field, type
- greatlakes-xfer.arc-ts.umich.edu
- armis2-xfer.arc-ts.umich.edu
- lighthouse.arc-ts.umich.edu
- Select “Interactive” for Logon Type
- In the User field, type your uniqname
- Click “Connect”
- Enter your Kerberos password
- Select your Duo method (1-3) and complete authentication
- Drag and drop files between the two systems
- Click “Disconnect” when finished
On Windows, you can also use WinSCP with similar settings, available alongside PuTTY here.
SFTP: Mac or Linux using Terminal
To copy a single file, type:
scp localfile [email protected]:./remotefile
To copy an entire directory, type:
scp -r localdir [email protected]:./remotedir
These commands can also be reversed in order to copy files from Great Lakes to your machine:
scp -r [email protected]:./remotedir localdir
You will need to authenticate via Duo to complete the file transfer.
Globus: Windows, Mac, or Linux
Globus is a reliable high performance parallel file transfer service provided by many HPC sites around the world. It enables easy transfer of files from one system to another, as long as they are Globus endpoints.
- The Globus endpoint for Great Lakes is “umich#greatlakes”.
How to use Globus
Globus Online is a web front end to the Globus transfer service. Globus Online accounts are free and you can create an account with your University identity.
- Set up your Globus account and learn how to transfer files using the Globus documentation. Select “University of Michigan” from the dropdown box to get started.
- Once you are ready to transfer files, enter “umich#greatlakes” as one of your endpoints.
Globus Connect Personal
Globus Online also allows for simple installation of a Globus endpoint for Windows, Mac, and Linux desktops and laptops.
- Follow the Globus instructions to download the Globus Connect Personal installer and set up an endpoint on your desktop or laptop.
Batch File Copies
A non-standard use of Globus Online is that you can use it to copy files from one location to another on the same cluster. To do this use the same endpoint (umich#greatlakes as an example) for both the sending and receiving machines. Setup the transfer and Globus will make sure the rest happens. The service will email you when the copy is finished.
Command Line Globus
There are Command line tools for Globus that are intended for advanced users. If you wish to use these, contact HPC support.