Follow up on the progress of Jobs, Jobs Steps and use of resources

The command "sacct" allows to obtain a lot of information on Jobs and their Steps. Official documentation : https://slurm.schedmd.com/sacct.html

- The following command displays the default information for Job # 1234 :

The first line corresponds to the entire Job and the following lines (JobID followed by a dot '.') Indicate the different Steps of the Job. Step "763.2" (3rd Step of Job 763) is here running (State: RUNNING).

- The option "--format" (or "-o", short version) allows to choose the fields to display and their size on the screen (by using '%'). Many attributes are available; consult the documentation for the complete list and their meaning

The information displayed here is chosen and the JobName field now displays the full name of the Job and Steps.

- Display of resource usage statistics (CPU / RAM / Disk ...) :

Note : Some attributes are only available once Step is complete.

Note 2 : It is possible to modify the date display format by modifying the environment variable SLURM_TIME_FORMAT. The date format used by Slurm is that of the "strftime" C function (http://man7.org/linux/man-pages/man3/strftime.3.html).

The above example uses the French date format (DD/MM/YYYY hh:mm:ss). To set the date format, the easiest way is to add the following line to the ".bashrc" file of your HOME user :

- Display for a Job Running Steps in Parallel :

Note : The allocation is here 18 CPUs instead of the previous 6 (3 Tasks of 6 CPUs are run in parallel).

- For those who would like to retrieve this information to process it in a script (and / or format it in another language), sacct has the options "--parsable" and "--parsable2" which return the same information but whose fields are separated by a "pipe" ("|"). In "parsable" mode, field sizes ("% ..") are useless, uncut values are always returned. The difference between these two options is that "--parsable" adds a "|" at the end of the line whereas "--parsable2", no.

Note : The "--noheader" option also allows you to not display the headers in the result.