As I Understand It — Episode 02: Making Sense of NVIDIA GPU Names Audio: https://sridharrajarao.com/audio/podcast/02-nvidia-gpu-names.m4a Transcript Lightly edited for readability. Timestamps are approximate. [00:00] I kept hearing these names H100, H200, B200, GB200, and I wanted to understand what they actually mean. [00:08] Welcome to As I Understand It. I am Sridhar. [00:12] At first I thought they were all different GPUs, but some names refer to a GPU, some combine a CPU and GPUs, and some describe an entire rack. Once I separated those, it started making more sense. Let me start with A100. A100 uses NVIDIA's Ampere architecture. Here architecture simply means the underlying design and generation. A100 became an important GPU for training and running AI models. It has specialized processing units called Tensor Cores, which help with the matrix calculations used heavily in AI. [00:50] But models kept getting bigger. They needed more computation, more memory, and faster connections. That brings us to Hopper, the next architecture I wanted to cover. The two names you will hear here are H100 and H200. H100 brought improvements in AI computation. H200 also belongs to the Hopper family, with a major improvement in memory capacity and bandwidth. Memory capacity is how much data you can hold. [01:24] Bandwidth is how quickly you can move it. That distinction helped me. A GPU can be very fast at calculation, but it still needs somewhere to hold the model and a way to feed data to its processing units. So when somebody says one GPU is better than another, I now ask better at calculation, memory, or both. Then there is GH200. This is where the naming gets confusing. GH200 means Grace Hopper Superchip. [01:59] It combines Grace CPU with the Hopper GPU and connects them through a fast link. So H200 and GH200 are different products. H200 is a GPU. GH200 combines a CPU and a GPU. Keep the distinction in mind because we see something similar with Blackwell. With Blackwell, the names to remember are B200 and GB200. [02:33] B200 is a Blackwell GPU. GB200 is a Grace Blackwell Superchip. It combines one Grace CPU with two Blackwell GPUs. So adding the G doesn't just mean it's a faster B200. We are talking about a different combination of components. Now take one more step, GB200 NVL72. That's a rack-scale system containing 72 Blackwell GPUs and 36 Grace CPUs. [03:09] NVLink connections and switches allow the GPUs to exchange data at high speed. This is the part I had to get clear in my head. B200 is a GPU. GB200 combines a CPU and two GPUs. GB200 NVL72 brings those building blocks together into a rack. If we compare their performance, we have to be clear about which level we are comparing. [03:41] After Blackwell comes Blackwell Ultra. Here you will hear B300 and GB300. B300 is a Blackwell Ultra GPU. GB300 brings Grace CPUs together with Blackwell Ultra GPUs in an integrated platform. Again the improvements involve computation, memory and effectively the whole system works together. And following Blackwell Ultra is Rubin. [04:14] The GPU is called Rubin, the associated CPU is called Vera. Together they form the Vera Rubin platform. That gives me a sequence I can remember. Ampere, Hopper, Blackwell, Blackwell Ultra, Rubin. But I don't think remembering every product name is the main thing. What matters is understanding what the workload needs. If you are training a large model, you may need many GPUs working together. [04:47] If you are serving responses to users, you also care about response time, how many requests you can handle, and the cost of running the system. So the newest GPU isn't automatically the right answer for every job. The questions I keep asking are… How much computation can it do? How much memory does it have? And how fast can it move data? How does it connect to the other processors? That's where I've got to with these names. [05:21] Now when I hear GB200 or GB300, I can place it in the bigger picture. And when someone says their AI hardware is faster, my next question is, faster at what? And are we comparing a GPU, a server, or a whole rack?