AI data centers: Why GPU racks need liquid cooling and so much power

An AI data center is a building packed with thousands of specialized chips, mainly graphics processing units (GPUs), that train and run AI models. It looks like an ordinary data center from the outside. Inside, the chips draw far more electricity and give off far more heat per square foot, which changes how the whole site is powered and cooled.

Why a rack of GPUs is a different kind of machine

Think of a hair dryer. One is harmless. Put a hundred in a closet and the closet needs its own wiring and a way to move the hot air out. That is the shift from traditional servers to AI hardware.

Servers sit in tall metal frames called racks. A conventional rack has historically drawn a few kilowatts. Nvidia’s GB200 NVL72 packs 72 GPUs into one rack and is rated at about 120 kilowatts (kW), according to The Register’s 2024 analysis. That figure leaves out the power for the pumps that cool it.

The reason is that these chips work as one unit. Training a large model means thousands of GPUs exchanging data constantly, so builders place them as close together as possible with fast links between them. Closeness saves time but concentrates heat.

Moving heat with water instead of air

Air is a poor carrier of heat. Beyond a certain density, fans cannot move enough of it to keep a chip safe. The GB200 NVL72 is therefore sold as a liquid-cooled rack. The Register notes that each superchip’s two accelerators are rated at 1,200 watts each, and that the design requires liquid cooling.

The common approach is called direct-to-chip cooling. A metal plate sits on each chip, and coolant flows through it and carries the heat away to a coolant distribution unit (CDU). The CDU passes that heat to the building’s own water loop, which sends it outside.

Liquid also saves energy in a small way. The Register points out that an air-cooled system can spend 15 to 20 percent of its power on fans, which a liquid design largely avoids.

• Air cooling: simple and familiar, but limited at high rack power.
• Direct-to-chip liquid: handles most of the heat, with some air still used for other parts.
• Immersion: servers sit in a bath of non-conductive fluid; less common today.

Where the electricity comes from

Cooling is only half the problem. The other half is getting power to the building. The International Energy Agency (IEA) said in its 2025 report Energy and AI that data centers used about 415 terawatt-hours (TWh) of electricity in 2024, roughly 1.5 percent of world consumption, with the United States at 45 percent and China at 25 percent. The agency projects that this will more than double to around 945 TWh by 2030.

The IEA adds that a typical AI-focused data center uses as much electricity as 100,000 households, and the largest ones now under construction will use 20 times that. Globally, data centers account for about one-tenth of electricity demand growth to 2030, but in the United States nearly half. These are projections, and the IEA’s own range for 2035 runs from 700 to 1,700 TWh.

Because these sites are so large and so concentrated in a few regions, they strain local power grids. Utilities must build new lines and generation, and developers sometimes add their own power plants.

How the buildout reached this scale

Cloud providers have run large data centers for two decades. The change came with deep learning, when researchers found that bigger models trained on more compute kept improving. That made GPU clusters the main product that cloud and AI companies compete on.

Nvidia then began selling whole racks and data center designs, not only chips. Its Blackwell generation, announced in 2024, was built around the NVL72 rack. Other chipmakers and cloud providers have their own rack-scale designs, so the pattern is industry-wide.

The buildings are mostly paid for by a few companies. OpenAI is one: our report on its reported $1.25 trillion valuation notes that the fundraising comes as public opinion has turned against AI companies and data center projects, per The Verge’s summary of Bloomberg. That backlash shows how a technical choice, packing more power into each rack, becomes a local question about land, water and electricity bills.

Limits, trade-offs and what to watch

Each step up in density has costs. Liquid cooling needs plumbing, leak detection and new skills for technicians. Older buildings designed for air often cannot host these racks without major rework. A rack that draws 120 kW also needs heavier electrical gear.

Water use is a separate point of dispute. Systems that reject heat by evaporating water can consume a lot of it, while closed loops use much less. Vendor claims about water and energy savings come from the companies selling the equipment, so independent measurements matter.

Power is the harder limit. Grid connections can take years, which is why builders look at sites near existing generation. Flexible operation, where a site reduces its load at peak times, is one response, though the IEA notes that AI sites are costly to switch off because the hardware is so expensive.

• Rack power: newer Nvidia designs are reported to go higher than 120 kW per rack.
• Efficiency: whether chips deliver more work per watt as fast as demand grows.
• Local rules: permits, water limits and community opposition.
• Utilization: whether new capacity is filled, which depends on demand for AI services.

Anyone reading headlines about gigawatt campuses is, in effect, reading about these same constraints: heat that must be removed, and power that must arrive.