Design a small GPU deployment
Design a small service on paper. The goal is to connect demand, capacity, and failure behavior, not to buy equipment.
Can the small service lose a server
6 replicas × 20 = 120 requests/s
4 replicas × 20 = 80 requests/s
At least 50 acceptable requests/s
The fictional requirement
Each independent GPU replica fits in 24 GB including working overhead and delivers 20 acceptable requests per second.
Demand peaks at 50 requests per second. Losing one whole server must not drop capacity below that.
Three servers with two replicas each are one candidate: 120 requests per second normally, and 80 after losing a server.
Check the assumptions
Those rates assume comparable requests, effective routing, and adequate CPU, storage, network, and memory.
A real 24 GB GPU does not automatically achieve the invented rate. Use measured workload results and a supported chassis.
Your deliverable
Draw the three servers and request routing. List software compatibility, model loading, power, cooling, access control, and acceptance tests that remain to be verified.
Check your answer
Two servers with two replicas each leave only 40 requests per second after one server fails. Three leave 80. The arithmetic supports a candidate design, not proof that its dependencies work.