Skip to content

Basic Commands

These are some of the basic commands:

  • sinfo – Quick view of hardware allocated and free
  • smap – More visual version of sinfo using ncurses
  • sview – Graphical interface of hardware (requires X11).
  • sbatch – Submit a job file
  • squeue – View all running jobs
  • squeue -u – View particular ’s jobs (could be you)
  • sshare – View fairshare information
  • sprio – View queued job’s priority

If you are a PBS Torque user and want to migrate to Slurm, you can find the equivalent examples for PBS and Slurm job scripts in the following table.

Command PBS/Torque Slurm
Job submission qsub job_script sbatch job_script
Job submission qsub -q queue -l nodes=1:ppn=16 -l mem=64g job_script sbatch --partition=queue --nodes=1 --cpus-per-node=16 --mem=64g job_script
Node count -l nodes=count --nodes=1
Cores per node -l ppn=count --cpus-per-node=count
Memory size -l mem=16384 --mem=16g
Wall clock limit -N name  --job-name=name

The sbatch arguments here are the minimal subset required to accurately specify a job on the h2p cluster. Please refer to man sbatch for more options.

SBATCH ARGUMENT DESCRIPTION
--nodes Maximum number of nodes to be used by each Job Step.
--tasks-per-node Specify the number of tasks to be launched per node.
--cpus-per-task Advise the Slurm controller that ensuing job steps will require a certain number of processors per task.
--error File to redirect standard error
--job-name The job name.
--time Define the total time required for the job
The format is days-hh:mm:ss.
--cluster Select the cluster to submit the job to smp, mpi and gpu are the available partition in the H2P
--partition Select the partition to submit the job to. smp, high-mem for smp cluster, opa, legacy for mpi cluster, gtx1080, titan, titanx and k40 for gpu cluster.
--account Charge resources used by this job to specified account. This is only relevant for users who are in multiple Slurm accounts because he/she is in groups that are collaborating.

srun also takes the --nodes,--tasks-per-node and --cpus-per-task arguments to allow each job step to change the utilized resources but they cannot exceed those given to sbatch. The above arguments can be provided in a batch script by preceding them with #SBATCH. Note that the shebang (#!) line must be present. The shebang line can call any shell or scripting language available on the cluster. For example, #!/usr/bin/env bash.

Slurm is very explicit in how one requests cores and nodes. While extremely powerful, the three flags,--nodes, --ntasks, and --cpus-per-task can be a bit confusing at first. 

--ntasks vs. --cpus-per-task
The term “task” in this context can be thought of as a “process”. Therefore, a multi-process program (e.g. MPI) is comprised of multiple tasks. In Slurm, tasks are requested with the --ntasks flag. A multi-threaded program is comprised of a single task, which can in turn use multiple CPUs. CPUs, for the multithreaded programs, are requested with the --cpus-per-task flag. Individual tasks cannot be split across multiple compute nodes, so requesting a number of CPUs with --cpus-per-task flag will always result in all your CPUs allocated on the same compute node.