Architecture · Simulation
Why AI racks run hot
Move the compute intensity from a traditional data centre to an AI one and watch what has to change: more accelerators, more power, more heat, and a different way of cooling and connecting the rack.
AI data centre. Racks full of accelerators: liquid cooling and a dedicated fabric become essential.
Accelerators
60
per rack
Power in
106kW
traditional ~8 kW
Heat to remove
104kW
almost all of the power
Cooling
Direct liquid
A traditional rack is often around 5–15 kW. Rack-scale AI systems draw about 120 kW (GB200 NVL72, SemiAnalysis 2024) to 135 kW (GB300 NVL72, Lenovo 2026), with peaks of up to 155 kW; the far end of the slider is a GB300 NVL72-class rack. The rack drawing is schematic.
CPU servers · air cooled · moderate density
60 accelerators · direct liquid · very high density
InfiniBand / high-performance Ethernet between racks; NVLink inside them. AI workloads spread one job across many accelerators, so the network carries an enormous amount of traffic.
Why it happens
- More accelerators
- More compute
- More power
- More heat
- More cooling
- Specialised infrastructure
Almost all the electrical power that computing equipment consumes eventually becomes heat, and that heat has to be removed. Higher rack power density means higher heat density, which needs more capable cooling.
What the rack now needs
- GPU / accelerator
- GPU memory
- NVLink
- Power distribution
- Liquid cooling
- High-speed networking
- Storage
AI data centres are different because they concentrate enormous amounts of compute into a small physical footprint. An AI rack is not a conventional rack with GPUs added: the power, cooling and network around it change too.
How it works
Compute. AI workloads need enormous amounts of parallel computation, so racks fill with accelerators (GPUs) instead of general-purpose CPU servers. More accelerators means more compute per rack.
Power. More computation draws more electrical power. The rack's power density rises well beyond what a traditional data hall was designed to deliver.
Heat. Almost all of that power ends up as heat. That is physics, not inefficiency: the energy used to compute has to go somewhere.
Cooling. Air cooling copes at traditional densities. As heat density climbs, coolant is piped to cold plates on the processors, warms up, and returns to a coolant distribution unit (CDU) that passes the heat to the building's heat-rejection plant.
Network. One training job is spread across many accelerators that constantly exchange data: NVLink inside a rack-scale system, InfiniBand or high-performance Ethernet between racks.
All figures here are illustrative, chosen to show the relationship, not to specify any product.