Racks pulling 50–100 kW force new cooling in AI data centers

The AI boom is colliding with the physical limits of the grid, heat and data links. Data center operators are finding that traditional cooling and copper interconnects can’t keep up with the pace of AI deployment, and that grid infrastructure isn’t expanding fast enough to support new capacity. The result is a mix of higher costs, slower rollouts and more innovation in how these mega clusters are cooled and connected.

Racks filled with GPUs are pulling ever more power. In many modern AI setups, a single cabinet can consume 50 to 100 kilowatts. That kind of heat demands more aggressive cooling and, as temperatures rise, performance can throttle back to protect hardware. In simple terms: when the equipment gets too hot, it slows down to prevent damage. That kind of throttling chips away at the gains AI teams are chasing.

Traditional air and liquid cooling are reaching their practical limits. Air cooling struggles to move enough air through densely packed racks, and liquid cooling, while more effective, requires a lot of pumping power and careful design to avoid leaks and inefficiencies. Operators are turning to newer approaches, including direct-to-chip cold plates, two-phase immersion cooling, and closed-loop liquid systems. These methods can cut cooling energy use, but they’re costly and slow to scale across thousands of racks.

Interconnects, the wires that shuttle data between GPUs, boards and servers, are hitting a wall too. Copper traces and cables lose strength over distance and generate electrical noise, especially as bandwidth demands surge. The industry is increasingly leaning on optical interconnects and silicon photonics to move data with less loss, even if those options come with higher upfront costs. For some facilities, the payoff is worth it because it reduces the need for power-hungry cooling and keeps performance steady.

Beyond the data center floor, the electric grid itself isn’t growing fast enough to match AI’s appetite. Operators report that grid and substation constraints limit when and where new AI capacity can come online. This grid strain means slower builds and, in some cases, higher prices for power as operators compete for scarce resources during peak demand. In short, the grid is as much a bottleneck as the equipment inside the racks.

Microsoft and other operators have begun testbeds and new designs to address these bottlenecks, but mass adoption will take time. The economics of cleaning up heat and boosting power efficiency are closely tied to the pace of AI innovation itself. If you’re wondering what this means for your tech, it’s mostly about timing and cost: AI services could see steadier prices and occasional slowdowns during peak load, even as efficiency improves at the component level.