As I Understand It — Episode 05: Cooling AI: Following the Heat Audio: https://sridharrajarao.com/audio/podcast/05-cooling-ai-following-the-heat.m4a Transcript Lightly edited for readability. Timestamps are approximate. [00:00] Last episode we followed electricity from the grid to the GPU. Now I wanted to understand, where does the heat go? I kept seeing words like cold plate, coolant and CDU. What helped me was following the heat through the system, one step at a time. [00:21] Welcome back to As I Understand It. I'm Sridhar. [00:25] Let's start with the GPU doing calculations. It gets hot. A metal cold plate sits on top of it, and liquid flows through the channels inside the plate. Heat moves from the GPU through the metal into the liquid. The coolant leaves warmer carrying the heat away. So where does it go? To a CDU, a coolant distribution unit. For this explanation, I am following a liquid-to-liquid cooling design. [00:59] Other arrangements exist. Inside this CDU is a heat exchanger. On one side, you have the warm coolant returning from the GPUs. On the other side, you have cooler water from the building's cooling system. And between them, a thin metal wall. That was the part I needed to understand. How can heat move across if the liquids don't mix? Think of holding a hot cup of tea. [01:31] Heat travels through the cup into your hand. Your hand never touches the tea. The heat exchanger works on that same basic principle. Heat passes through the metal from the warmer GPU coolant into the cooler building water. The heat crosses. The liquid stays separate. Now, let's finish the first loop. The GPU coolant has given up some heat. A pump circulates it back to the cold plates. [02:04] It picks up more heat, returns to the CDU and goes around again. That's loop one. But now the building water is warmer. How does that get cool? It travels to the facility's cooling equipment, which transfers the heat outside. One option is a dry cooler, similar to a car radiator. Water flows through the tubes, and fans move outside air across them to carry heat away. When the outside conditions are unsuitable, a chiller can use refrigeration to cool the water. [02:44] Some designs use cooling towers, where evaporation helps release the heat. These systems can involve additional circuits. Once cooled, the building water is pumped back to the CDU to collect more heat. That's loop two. So, let me put both together. GPU coolant collects heat from the chip and delivers it to the CDU. Building water collects that heat at the CDU and carries it to the cooling equipment. [03:17] Both liquids return and repeat their journeys. The heat keeps moving forward. And the coolant doesn't have to feel ice cold. It needs to be cool enough to absorb heat while keeping the chips within their operating limits. Its chemistry, cleanliness, and flow matter too. But now I have the picture. Two circulating loops, one heat handoff and a path outside. That's cooling an AI data center, as I understand it.