How to get started Slurm job environment variables from the master node?
Apr 2, 2:30pm

When working with Slurm, the job environment variables available to your job depend on when and how the job is submitted. Here’s how you can access and understand Slurm job environment variables from the master node (the login node where you submit jobs):


1. Slurm Environment Variables Available During Job Execution

When Slurm runs your job, it sets several environment variables that describe the job’s context. These are available inside the job script on the compute nodes.

Common Slurm Environment Variables

SLURM_JOB_ID
The unique job ID assigned by Slurm.
SLURM_JOB_NAME
The name of the job (as specified with --job-name).
SLURM_JOB_NODELIST
List of nodes allocated to the job.
SLURM_JOB_NUM_NODES
Number of nodes allocated.
SLURM_NTASKS
Number of tasks (processes) allocated.
SLURM_NPROCS
Total number of CPUs allocated.
SLURM_CPUS_PER_TASK
CPUs per task (if specified).
SLURM_MEM_PER_NODE
Memory per node (if specified).
SLURM_PARTITION
Partition (queue) the job is running in.
SLURM_TIMELIMIT
Time limit for the job.
SLURM_WORKING_DIR
The working directory from which the job was submitted.
SLURM_SUBMIT_DIR
The directory where the job script was submitted.
SLURM_EXIT_CODE
Exit code of the job (if available).

2. How to Access These Variables

From a Job Script

If you submit a job using a script (e.g., sbatch), you can access these variables directly in your script:

bash
#!/bin/bash
#SBATCH --job-name=my_job
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --mem=8G
#SBATCH --time=01:00:00

echo "Job ID: $SLURM_JOB_ID"
echo "Job Name: $SLURM_JOB_NAME"
echo "Node List: $SLURM_JOB_NODELIST"
echo "Number of Nodes: $SLURM_JOB_NUM_NODES"
# ... and so on

From the Command Line (Master Node)

You can also check the environment variables of a running job from the master node using:

bash
scontrol show job <JOB_ID> -o

or

bash
squeue -j <JOB_ID> --format="%i %j %N %P %T %M %D %m %c"

3. Submitting Jobs with Environment Variables

If you want to pass environment variables from the master node to the job, you can use:

bash
export MY_VAR="value"
sbatch --export=ALL,MY_VAR my_script.sh

or

bash
sbatch --export=MY_VAR1,MY_VAR2 my_script.sh

4. Accessing Variables from Within a Running Job

If you want to inspect the environment of a running job, you can use:

bash
srun --jobid=<JOB_ID> env

Summary Table: Key Slurm Environment Variables

SLURM_JOB_ID
Unique job ID
SLURM_JOB_NAME
Job name
SLURM_JOB_NODELIST
Nodes allocated
SLURM_NTASKS
Number of tasks
SLURM_NPROCS
Total CPUs
SLURM_PARTITION
Partition name
SLURM_TIMELIMIT
Job time limit
SLURM_WORKING_DIR
Working directory

Example: Print All Slurm Variables

bash
#!/bin/bash
#SBATCH --job-name=env_test
#SBATCH --nodes=1
#SBATCH --ntasks=1

echo "All Slurm environment variables:"
env | grep SLURM

Let me know if you want to see the output of a specific command or need help with a concrete example!

Apr 2, 2:30pm

This is a shared conversation and may include unverified content that doesn't reflect Mistral AI's views. Login to resume chatting.