A chip does its work by switching electricity on and off billions of times a second. Almost all of the electricity it uses ends up as heat: the energy has to go somewhere, and in a chip it leaves as warmth. An AI chip uses a great deal of electricity in a very small space, so it makes a great deal of heat in that space. If the heat is not carried away as fast as it is made, the chip gets hotter until it has to slow down or stop. A cooling system is what keeps the heat moving out, every second the chip is running.
Heat is the by-product of computing
Every computer turns electricity into heat. A laptop has a small fan for this reason. A data centre does the same thing at a much larger scale, and an AI data centre does it with far more heat packed into each cabinet of equipment.
Engineers call one cabinet of equipment a rack. The joint AI Data Center Energy Performance Framework from PNNL, ASHRAE and NEMA notes that racks of graphics processors used for AI often reach 40 to 100 kW per rack, compared with 5 to 10 kW in older racks built around ordinary processors. That is roughly ten times the heat in the same floor space.
Why the heat cannot simply be left alone
Electronic equipment is designed to work within a temperature range. For the air entering servers, the US Department of Energy's best practice guide, written by NREL, cites ASHRAE's recommended range of 18°C to 27°C. Outside the allowable range, equipment is at risk, and modern chips protect themselves by slowing down when they get too hot. A slowed chip means a slower answer and wasted investment, so cooling is part of the computer, not an extra.
The heat also does not stop. An AI chip running a training job or answering questions produces heat for as long as it runs, day and night. The cooling has to match that load continuously.
Why air is giving way to liquid
For decades, data centres cooled servers by blowing cool air through them. Air works well when the heat is spread out. When it is concentrated, moving enough air becomes hard: it takes more fans, more power and more space.
Liquid carries heat far better. The same NREL guide puts it simply: water flow is a much more efficient method of transporting heat. The PNNL, ASHRAE and NEMA framework describes direct-to-chip liquid as an inherently more efficient heat transfer medium than air.
This is why the newest AI systems are designed around liquid. NVIDIA describes its GB200 NVL72 as connecting 36 Grace CPUs and 72 Blackwell GPUs in a rack-scale, liquid-cooled design. Its technical guide explains that the processors and the main network chips are liquid cooled, while the remaining components are still cooled by air. Most AI racks today are hybrids of this kind.
What a cooling system has to do, step by step
| Step | What happens | Where it happens |
|---|---|---|
| Capture | Heat leaves the chip into a metal cold plate with liquid flowing through it, or into air blown across it | Inside the server |
| Carry | The warmed liquid or air moves away from the chip | Inside the rack and the hall |
| Hand over | A heat exchanger passes the heat to the building's cooling loop | In the hall or plant room |
| Reject | The building releases the heat outdoors, to air or to water | Outside the building |
The companion article, How Do AI Data Centres Actually Stay Cool?, explains the last two steps and the choices engineers make there.
What is not certain
- Rack figures vary widely by design. The 40 to 100 kW range is a guide from the PNNL, ASHRAE and NEMA framework, not a measurement of any one facility.
- How hot a chip may safely run depends on the chip and its maker. This article gives ranges for the air entering servers, not limits for any specific chip.
- Product descriptions come from the manufacturer and describe one system design. Other designs differ.
Sources and method
- PNNL, ASHRAE and NEMA, AI Data Center Energy Performance Framework: Energy and Thermal Efficiency. Undated web page, read 29 October 2026. https://www.ashrae.org/technical-resources/topics-and-initiatives/ai-data-center/energy-and-thermal-efficiency
- Van Geet and Sickinger, NREL, for the US Department of Energy Federal Energy Management Program. Best Practices Guide for Energy-Efficient Data Center Design, revised July 2024. https://www.energy.gov/sites/default/files/2024-07/best-practice-guide-data-center-design_0.pdf
- NVIDIA, GB200 NVL72 product page, updated 27 August 2026. https://www.nvidia.com/en-us/data-center/gb200-nvl72/
- NVIDIA, GB200 NVL Multi-Node Tuning Guide, Understanding Your Grace-Blackwell Systems, last updated 21 April 2025. https://docs.nvidia.com/multi-node-nvlink-systems/multi-node-tuning-guide/system.html
Method: each link was opened on 29 October 2026 and every figure was taken from the page cited beside it. Temperatures are as printed in Celsius in Table 3-1 of the guide. No figure was estimated.
Read next
This article provides general information, not investment advice or an offer of securities.