Comment by Animats
4 hours ago
Next, plumbers. The future of data centers seems to be more liquid cooling and less ductwork. Take a look at this new 1 megawatt server rack.[1] It has more pipes than cables. All that coolant flow and all that electrical power in a crowded space is scary.
Listen to the video again, he clarifies early on that this is supporting multiple racks. A megawatt is 1000kW. If you took 5 B300 systems (10 rack units each), you'd still only be at 75kW, which if you're doing redundancy properly, means you have wired 150kW to the rack, but you're not generating a megawatt of heat.
NVidia has been talking about 800VDC 1MW racks for about a year now.[1] They're trying to get others to solve the power and cooling problems. The Schneider Electric view: [2] The view from Flex.[3]
This was being talked about more in 2025 than it is now. Nobody seems to have deployed a 1MW rack yet. There are major problems to be solved for that level of power density. The fire protection people want racks in fireproof enclosures with a built in inert gas extinguishing system and smoke detectors that pull in air from many points for very fast detection. The cooling people talk about 400 gallons per minute of chilled glycol. The power people talk about arc flash protection and worker safety.
This is a scary level of energy density. Not hopeless; the switchgear and semiconductor inside locomotives are at comparable levels.
[1] https://developer.nvidia.com/blog/nvidia-800-v-hvdc-architec...
[2] https://blog.se.com/datacenter/2025/10/16/the-1-mw-ai-it-rac...
[3] https://flex.com/resources/megawatt-racks-are-coming-how-800...
> The power people talk about arc flash protection and worker safety.
800VDC is insanely scary stuff.
> At present, no consensus standard provides a validated methodology for calculating direct current (dc) arc-flash incident energy.
This means that establishing proper PPE for DC arc flashes is not possible right now. If I was asked to provide electricians to work on these racks, I would only allow my electricians to approach them, let alone do any work, if they were de-energized, full stop. Even inspecting anything while they’re energized (without doing any work) would be insane, in my opinion. Nobody’s life is worth losing over a rack of servers, and the incident energy for a 480V AC circuit that can power a 1 MW rack (1503 amps) can exceed 40 cal per cm^2 depending on the electrical system already, and there is no PPE that protects you above that incident energy level.
> All that coolant flow and all that electrical power in a crowded space is scary.
Maybe it's a non-conductive oil? I recall seeing someone who overclocked their PC (circa 2005) to 5GHz, and ran the motherboard completely submerged in an oil bath.
Each rack has a water (and glycol?) loop and water blocks mounted to the chips with a liquid-liquid heat exchanger that interfaces with the building chilled water loop (glycol and water), there’s no immersion.
It works like a nuclear power plant cooling system if you want to look an a diagram that is easily available. The radiated core water loop is equivalent to the rack water loop and the cooling water piping is equivalent to the chilled water loop, the difference being that data centers recirculate their chilled water loop water while nuclear plants generally discharge the water after it has passed through cooling towers.
expensive and leaks become more problematic.