Growing Data Center Power Density and Improved Energy Efficiency

 

AI workloads require substantial amounts of power. Resultingly, AI chips are becoming more powerful and packed together more tightly. In data centers, growing demand for HPC (high performance computing) and AI workload will push rack densities well beyond the 15-25 kW seen today. According to IDTechEx's "Sustainability for Data Centers 2027-2037: Green Technologies, Market Forecasts, and Players" report, semiconductor manufacturers and data center cooling solution providers are predicting AI GPU peak rack power densities could skyrocket to 1 MW by 2030. This is triggering fundamental shifts in data center design - such as transitions to liquid cooling (direct-to-chip or immersion) and 800VDC architecture - to deliver improved performance and higher energy efficiency.

 

 

With the increasing demand for high-performance computing in sectors like AI, cloud computing, and crypto mining, the thermal design power (TDP) of chips has risen significantly over recent years. In 2020, the TDP of data center GPUs was below 500W. In comparison, for Nvidia's next-generation Rubin chip, the chip power is expected to exceed 2300 W (and rack power density expected to approach 400 kW). At this level, higher efficiency liquid cooling becomes more viable than traditional air-cooling.

 

 

 

 

Liquid cooling solutions include single-phase direct-to-chip cooling, single phase immersion cooling, two phase direct-to-chip cooling, and two phase immersion cooling. According to IDTechEx's analysis in its "Thermal Management For Data Centers 2026-2036" report, the general trend is that direct-to-chip cooling is more likely to be adopted with increasing rack density, and adoption of immersion cooling will remain niche into the 2030s (due to challenges such as high upfront costs (in terms of US$/Watt), maintenance, and complexities of retrofitting the server boards). From a sustainability standpoint, switching to liquid cooling is reported to cut overall data center CO₂ emissions by ~15%.

 

 

Today's racks in AI data centers rely on 54VDC power distribution. However, as identified by NVIDIA, racks are beginning to exceed 200 kW and subsequently starting to hit physical limits including "space constraints, copper overload, and inefficient conversions". The solution proposed by the data center sector is a transition to 800VDC. In this way, the data center power architecture is simplified (improving energy efficiency) and future-proofed for higher power delivery in the same space.

 

 

For this high voltage DC approach, the power electronics for data centers landscape is shifting (analyzed further in IDTechEx's "Power Electronics Market 2026-2036: Data Centers, Electric Vehicles, and Renewables" report). Some power conversion stages will be eliminated, and others will require more advanced semiconductor materials. Within conventional data center PSUs, power electronics using traditional silicon transistor architectures perform power factor correction and isolation/DC-DC conversion. But at higher voltages, the efficiency and size benefits for wide bandgap semiconductor materials such as SiC and GaN are much more favorable than Si. Additionally, IDTechEx has also observed interest in more nascent solutions to support 800VDC emerging in 2026, such as HTS (high temperature superconductor) tapes for power distribution in data centers to minimize transmission losses.

 

 

Since global electricity grids still use large amounts of fossil fuels, as data centers demand more power, there will inevitably be an increased carbon footprint relating to electricity generation. However, energy is increasingly being used more efficiently within data acenters. The adoption of liquid cooling in data centers offers improved PUE (power usage effectiveness) compared to conventional air cooling. Similarly, for 800VDC, the total efficiency from utility power to server-level PSUs (power supply units) could be boosted to 95%. Therefore, this higher energy efficiency combined with lowering carbon intensity of electricity grids, can help mitigate the negative environmental impact of AI according to IDTechEx's "Sustainability for Data Centers 2027-2037: Green Technologies, Market Forecasts, and Players" report.

 

 

For more information on this report, including downloadable sample pages, please visit www.IDTechEx.com/SDC, or for the full portfolio of sustainability and data center-related research available from IDTechEx, see www.IDTechEx.com.