As I Understand It — Episode 04: Powering AI: From the Grid to the GPU Audio: https://sridharrajarao.com/audio/podcast/04-powering-ai-grid-to-gpu.m4a Transcript Lightly edited for readability. Timestamps are approximate. [00:00] Last episode, I said that an AI data center needs a lot of electricity. Then I thought, okay, but how does the electricity actually reach the GPU? And if the building has generators, why does it also need batteries? [00:18] Welcome back to As I Understand It. I'm Sridhar. [00:23] This time, I went a little deeper into power. I looked at electrical designs and papers from Schneider Electric, Eaton, Vertiv, and researchers working on large AI systems. Here is how I now understand it. Electricity comes from the utility at a higher voltage. Transformers bring that voltage down for use inside the facility. Along the way, there is switchgear, equipment that controls electrical circuits and disconnects them when protection detects a problem. [01:00] Then distribution equipment carries power towards the racks. At the rack or server, power supplies convert AC, alternating current, into DC, direct current. More conversion brings it down to what the chips need. The exact arrangement varies, but that helped me picture the journey. Now, what happens if the utility power disappears? My first thought was, start the generators, but starting a generator takes time. [01:39] It has to become ready before it can supply the equipment. The GPUs can't just pause and wait for electricity. That gap is where UPS—an uninterruptible power supply—helps. Its stored energy keeps the protected equipment running while generator power becomes available. So the UPS bridges the interruption. The generator supports a longer outage. That distinction finally made sense to me. [02:11] But the part I found most interesting happens while everything is running. During a large AI training job, thousands of GPUs can work in step. They do calculations, then exchange information to coordinate their work. Their power demand can change between those activities. Now imagine thousands of them making that change together. The demand rises, drops, then rises again. [02:47] A paper from Microsoft, OpenAI, and NVIDIA looks at this using observations from real training systems. And Vertiv describes a UPS test with two seconds at 90% of rated power, followed by two seconds at 50% repeatedly. They tested the changing load on utility power, generator power and during the transfer. That helped me understand something. You need enough electrical capacity. [03:23] You also need equipment that can handle how the load behaves. Even carrying power through the rack becomes a challenge. For the same amount of power, higher voltage means lower current. That can reduce distribution losses and the amount of copper needed. That's one reason the industry is developing 800-volt DC systems. The GPU still needs much lower voltages. [03:58] The change is in how electricity gets closer to it. So now, when I hear about a data center needing so many megawatts, I picture the whole journey, getting the electricity, converting it, distributing it, protecting it, and keeping it steady while the GPUs keep changing what they need. Next time, I want to follow what happens after that electricity becomes heat. [04:36] That's powering AI, as I understand it.