Reading Academic Computing Cluster – GPU computing
New: Academic Computing Drop-in Sessions
Academic computing drop-in sessions are now run by members of our team on Wednesday afternoons! Head to our drop-in info page for details & schedule....
What is GPU computing?
GPU computing is the use of a GPU (graphics processing unit), which typically handles computation only for computer graphics, as a co-processor to accelerate CPUs (central processing units) for general scientific computing.
Some parallelised applications can run faster as the GPU accelerates applications running on the CPU by offloading some of the compute-intensive and time consuming portions of the code while the rest of the application still runs on the CPU.
A CPU consists of four to eight CPU cores, while the GPU typically consists of hundreds of smaller cores which is what gives the GPU its high compute performance. Some scientific applications and codes have GPU computing functionality. GPU-accelerated applications can be developed using the CUDA toolkit, which includes GPU-accelerated libraries, debugging and optimization tools.
GPU computing on the RACC
On the RACC, one can request one or more GPUs and combine it with a suitable number of CPUs. Let’s discuss this considering the following example script below, which can be found in /share/slurm_examples/gpu/.
#/bin/bash # standard CPU directives, tip: use cpus-per-task to allocate one or more cores per GPU #SBATCH --ntasks=1 #SBATCH --cpus-per-task=1 #SBATCH --threads-per-core=1 # plus the GPU line, you can request one or more (now up to 3) GPUs #SBATCH --gres=gpu:1 # partition 'gpu' or 'gpu_limited', or paying project partition #SBATCH --partition=gpu_limited # account 'shared' is default, there are also paying accounts #SBATCH --account=shared #SBATCH --job-name=example_gpu_job #SBATCH --output=gpu_out.txt #SBATCH --time=24:00:00 #(24 hours is the default in the partition 'gpu_limited') #SBATCH --mem=10G # in the 'gpu' partition rather use --mem=0 plus --exclusive ##SBATCH --mem=0 --exclusive #optional for diagnostics, shows the hostname and available GPUs hostname echo nvidia-smi echo echo CUDA_VISIBLE_DEVICES $CUDA_VISIBLE_DEVICES echo #and the actual job ./cuda_gpu.exe
The above is an example of a job where almost all the work is done on the GPU. We request just 1 CPU core (‘-cpus-per-task=1’). Allocating CPU and GPU cores does not always work like in the example above. Some jobs run both on CPUs and on GPUs, in which case it might be beneficial to allocate more CPU cores. GPUs are requested with the directive #SBATCH –gres=gpu:N, where N is the number of GPUs your job will use. In the example above we allocate just one GPU.
There are three options to access GPUs:
- The gpu partition: Those are older nodes (with Tesla K10 and Tesla P100) and small VMs providing access to relatively small GPU devices (Tesla M10). The main limitation of the VMs is the small amount of memory available. The advantage is that some of those GPUs are available for free with running time of up to 30 days.
- The gpu_limited partition: The gpu_limited partition provides time limited access to paid for project hardware. Currently the time limit is 5 days, but this will have to be reduced in the future. Some of the project nodes are equipped with one or more modern, high end GPUs and they have more memory available than the GPUs in option 1. However, the disadvantage is the time limit and the possibility that your free job can be suspended to give way to project jobs.
- Paying users can submit their jobs with specifying their project’s account name and partition name.
The free options are provided for testing. To have guaranteed access for the duration of the project, dedicated hardware or Research Cloud resources should be purchased (access can be via the Academic Computing Cluster, or it can be via a customized, dedicated VM).
When submitting GPU jobs to the ‘limited’ partition, you specify the memory allocation as usual, but please allocate as little memory as possible to limit the risk that your free jobs with clash with the hardware owner’s jobs. In the case of the ‘gpu’ partition, there is now always just 1 GPU per node and a rather small amount of memory per node. Your jobs will have the best chances to run if you submit them with the combination of options ‘–mem=0’ and ‘–exclusive’. With those options your job will not be limited by Slurm and you will have exclusive access to all node resources.
In our example we have added some lines that might help you to diagnose possible problems with accessing the GPUs. It is useful to know on which node your job runs, hence the ‘hostname’ command. Furthermore, the command ‘nvidia-smi’ shows information about the driver and all the GPUs. The existence of output from this command confirms that GPUs are present in the system and nvidia drivers are installed. Our example C program (in /share/slurm_examples/gpu/) also lists some information about the GPUs devices that are available to the program. Here, only the number of GPUs requested in the job script is shown, possibly not all the GPUs listed by nvidia-smi. CUDA programs obtain information about available devices from the environmental variable CUDA_VISIBLE_DEVICES. Slurm automatically sets this variable to indicate the GPUs allocated for the job. In your programs, please rely on this mechanism and do not specify device id explicitly.
The VM nodes in the ‘gpu’ partition, gpu-7-[3-6], are equipped with Tesla M10 GPUs and have 16GB of physical memory, the physical nodes are gpu-7-8 with Tesla P100 and gpu-7-9 with 8 x Tesla K10. You can request those with ‘–gres=gpu:M10:1’, ‘–gres=gpu:P100:1’, ‘–gres=gpu:K10:n’ (n<=8).
The nodes in the ‘gpu_limited’ partition have:
compute-7-0: Tesla V100, you can request this with ‘–gres=gpu:V100:1’
gpu-7-[1-2]: 3 x Tesla T4 in each node, you can request this with ‘–gres=gpu:T4:n’, where n is the number of GPUs.