SR

Learn with me · GPU course

From GPU to data center.

Start with the basics. Understand the hardware. Learn how to build and operate the system around it.

56 sessions in six parts, plus a getting-started lesson. Read in order like a book, or return to a chapter when you need it. Every session is available now.

Start with Session 1 → Session 0: identify your GPU

How to use this course

Each session has a short explanation, a diagram, and one exercise. Try it before opening “Check your answer.” Sources are tucked away under further reading when you want more detail.

Most exercises need only paper or a calculator. NVIDIA-specific commands require an authorized NVIDIA environment and the stated tools, not an Apple GPU. Hardware examples are documented configurations, not a newest-product ranking. Illustrative numbers are labeled.

Physical installation, electrical work, coolant handling, and disruptive tests require qualified teams and approved procedures. These lessons explain the decisions; they do not replace product manuals or supervised experience.

Part 1 · Sessions 1–10

Understand the GPU

From parallel work and memory to the execution units inside a GPU.

  1. Why do we need a GPU? Understand why GPUs exist and why they work alongside CPUs.
  2. Many hands, one job Recognize work that can run in parallel and work that must wait.
  3. Where does the work go? Distinguish working memory from storage and explain why capacity matters.
  4. Why does a GPU wait? Find the bottleneck before choosing faster hardware.
  5. What is inside a GPU? Identify the major hardware blocks without confusing cores with whole processors.
  6. How does a GPU organize work? Map a calculation to threads, blocks, and a grid.
  7. What is a warp? Understand grouped execution, divergence, and why synchronization matters.
  8. Why are there different kinds of memory? Distinguish registers, shared memory, caches, and device memory.
  9. What do specialized math units do? Understand matrix operations and the purpose of Tensor Cores.
  10. What does numerical precision mean? Connect number formats to memory, accuracy, and performance.

Part 2 · Sessions 11–19

Choose hardware and understand the server

Workloads, GPU types, server configurations, and the connections between devices.

  1. Training versus inference Describe how learning a model differs from using it.
  2. Which kind of GPU do you need? Choose a GPU class using workload requirements, not branding alone.
  3. How to read a GPU specification Separate per-device specifications, aggregate totals, and theoretical peaks.
  4. What does NVIDIA provide? Understand vendor responsibilities from silicon through operations.
  5. What is inside a GPU server? Trace the components surrounding the accelerators.
  6. How many GPUs belong in a server? Compare device counts with workload and topology requirements.
  7. How does PCIe connect everything? Understand link width, shared paths, and locality.
  8. How do GPUs talk to each other? Distinguish GPU interconnects from the server network.
  9. What makes a balanced server? Budget the resources needed to keep accelerators productive.

Part 3 · Sessions 20–29

Build the rack and data center

Space, power, cooling, networking, storage, and evidence that the system is ready.

  1. How is a GPU rack assembled? Treat rack layout as a capacity and serviceability problem.
  2. Where does the power come from? Understand power budgets, redundancy, and energy without attempting electrical work.
  3. How is GPU heat removed? Trace heat from the chip to the facility and identify monitoring boundaries.
  4. What is a rack-scale system? Distinguish an integrated GPU rack from independent servers sharing a cabinet.
  5. Why are there different networks? Separate management, workload, and storage traffic by purpose and risk.
  6. Ethernet, InfiniBand, and RDMA Understand what RDMA changes and what it does not fix.
  7. How do racks become a cluster? Calculate shared network capacity and identify failure boundaries.
  8. How do we feed the cluster? Connect storage behavior to GPU throughput and recovery.
  9. What must be ready before equipment arrives? Build a dependency-based readiness review.
  10. How do we accept a new cluster? Define evidence that the delivered system can run the intended workload.

Part 4 · Sessions 30–40

Run and operate it at scale

Software, scheduling, monitoring, troubleshooting, maintenance, and useful capacity.

  1. What software makes the GPU usable? Separate drivers, toolkits, libraries, and application environments.
  2. How does a job get its GPUs? Understand allocation, queues, and placement constraints.
  3. How can teams share a cluster? Separate quotas, time sharing, hardware partitioning, and security isolation.
  4. How does work spread across GPUs? Understand data, tensor, and pipeline parallelism.
  5. What should we monitor? Connect device health to workload and facility signals.
  6. Is the cluster doing useful work? Distinguish activity, allocation, throughput, and goodput.
  7. Why is this job slow? Use comparable evidence to isolate a performance regression.
  8. What happens when hardware fails? Separate error evidence, containment, and return-to-service decisions.
  9. How do we maintain a live cluster? Plan changes around workload safety, compatibility, and rollback.
  10. How do we manage incidents and recovery? Coordinate technical recovery around impact and recoverable work.
  11. How do we plan growth and cost? Calculate usable capacity and cost per useful outcome.

Part 5 · Sessions 41–44

Put the learning together

Worked design exercises and incident scenarios that connect the whole system.

  1. Design a small GPU deployment Produce a defensible first design from explicit workload assumptions.
  2. Expand it across racks Extend a design while preserving failure and dependency reasoning.
  3. Write the operating plan Turn a hardware design into an owned and testable service.
  4. Diagnose a troubled cluster Apply the course through three evidence-based investigations.

Part 6 · Sessions 45–56

When GPU infrastructure fails

Leaks, cooling faults, power, unreliable links, missing telemetry, and safe recovery.

  1. The rack is leaking Separate leak collection, leak detection, and the response that protects a rack.
  2. Who notices the leak Follow a leak signal from the sensor to an accountable responder.
  3. Hoses seals and connectors Understand why cooling connections need traceability and controlled maintenance.
  4. Coolant is flowing but is it enough Use flow, pressure, temperature, and workload together to investigate cooling.
  5. When the cooling unit fails Distinguish a component failure from losing cooling for a shared group of racks.
  6. The coolant itself matters Understand fluid compatibility, condition, and the evidence needed after maintenance.
  7. Too cold can also be a problem Understand dew point and distinguish condensation from a suspected coolant leak.
  8. One GPU keeps overheating Compare local thermal symptoms with shared cooling and workload evidence.
  9. The rack loses power Trace electrical dependencies and distinguish spare components from resilient service.
  10. The network works but training crawls Investigate physical-link evidence without confusing every slowdown with a bad cable.
  11. The monitoring goes quiet Recognize stale measurements and test the path from observation to notification.
  12. Can this rack return to service Combine physical repair, monitoring, and workload validation in a recovery decision.