How does a job get its GPUs?

Count and placement both matter

A job may require a GPU type, memory capacity, CPU resources, and several GPUs on the same server.

Four free GPUs scattered across four servers cannot satisfy a request for four on one server. This is fragmentation: free capacity exists in the wrong shape.

Some distributed jobs need their participants allocated together. That requires suitable scheduler support and configuration.

Different schedulers use different models

Slurm commonly manages queued batch work. Kubernetes schedules Pods and can expose GPU resources through device plugins and other supported mechanisms.

Whichever you use, distinguish physical GPUs from partitions or sharing slots. An allocation is not proof that useful work is running.

Try it yourself

Server A has three free GPUs. Server B has one. Can jobs needing two or four GPUs on one server start?

Check your answer

The two-GPU job fits on A. The four-GPU job cannot start under that constraint, even though four are free overall.

Sources and further reading

• GPU scheduling documentation
• Kubernetes GPU guide