Launcher
Launcher is a utility developed at the Texas Advanced Computing Center (TACC) to allow running of many serial or multi-threaded applications as a single, large batch submission.
Using Launcher
Launcher is available on Great Lakes as a software module. To enable it, load it as you would any other.
module load launcher
The module will add the installation directory to your $PATH as well as set the following environment variables:
LAUNCHER_DIR– installation directory. Only necessary for Launcher’s internal use.LAUNCHER_PLUGIN_DIR– plugin directory for integration with various resource managers. Only necessary for Launcher’s internal use.LAUNCHER_RMI– resource manager integration plugin to use.LAUNCHER_SCHED– scheduling method to use. Can be overridden by the user.
For more information on software modules, see the module documentation here.
Submitting a Launcher Job
*Please note: Launcher is not intended for MPI jobs.*
We will ultimately be using a job submission script as normal. For information on job submission syntax, see the Slurm User Guide.
First, set up your job file. This should be a plain text file with the commands that you need to execute, one command per line. For example:
python myscript.py input1
python myscript.py input2
python myscript.py input3
...
Assume the name of this file is $HOME/example-job-file. Then, in your job submission script, you would need to set the variable LAUNCHER_JOB_FILE to this value:
export LAUNCHER_JOB_FILE=$HOME/example-job-script
The last line of your submission script would be to call Launcher’s paramrun executable:
paramrun
This executable has been added to your path by the module. Tying everything together, your
job submission script for this run would look like this:
#!/bin/bash #SBATCH --job-name=Parametric #SBATCH --mail-user=[email protected] #SBATCH --mail-type=BEGIN,END #SBATCH --cpus-per-task=1 #SBATCH --nodes=2 #SBATCH --ntasks-per-node=2 #SBATCH --mem-per-cpu=4000m #SBATCH --time=10:00 #SBATCH --account=example #SBATCH --partition=standard #SBATCH --output=/home/%u/%x-%j.log module load launcher export LAUNCHER_WORKDIR=$PWD export LAUNCHER_JOB_FILE=$HOME/example-job-file $LAUNCHER_DIR/paramrun