How many GPUs belong in a server?
Eight GPUs can run eight independent jobs or work together on one job. The device count is the same; the communication requirements are not.
Eight GPUs in two arrangements
One chassis failure removes two GPUs
One chassis failure removes eight GPUs
Think about the failure boundary
Four two-GPU servers spread capacity across four chassis. One eight-GPU server concentrates it in one.
The larger server may offer better local connections for a shared job. The smaller servers may keep more independent work running after one chassis fails. Compare actual topology, not just counts.
A 72-GPU rack-scale system is yet another boundary, not an ordinary server with a bigger number.
Memory still needs a plan
Adding a GPU does not automatically enlarge a program’s local memory allocation. Software must support splitting the work or another suitable strategy.
Documented configurations such as DGX H100, L4 partner systems, and MI300X illustrate different options. They do not establish a universal “normal” count.
Try it yourself
Compare four two-GPU servers with one eight-GPU server. What does one chassis failure remove?
Check your answer
Two GPUs versus eight. For a tightly coupled job, also compare the cross-server communication cost and supported interconnect. Neither arrangement wins every workload.