Comment by why_at
3 hours ago
Maybe I'm being dumb, but I don't understand what the innovation is here.
I get that they're using liquid coolant at higher than usual temperatures, but why couldn't they do that before? Most of the comparison in the article is for air cooled datacenters but what about other liquid cooled ones?
Surely in all the previous datacenters that have been designed there has been someone doing the math and determining what temperature things need to run at, how much energy it will use, how much heat it all will produce, etc.
edit: just saw this:
>Previous liquid-cooled servers were hybrid: GPUs and CPUs got cold plates, but the rest of the system stayed air-cooled, with finned heat sinks designed to shed heat into moving air. In a fully liquid-cooled server, the cooling for these components needed to be completely redesigned to use liquid.
> Surely in all the previous datacenters that have been designed there has been someone doing the math and determining what temperature things need to run at, how much energy it will use, how much heat it all will produce, etc.
It seemed like a pretty big deal ~ 2011 when big companies were running their (air cooled) datacenters closer to 95F (35C) vs the traditional 72F (22C). So jumping up a little more is maybe not super exciting, but it's still innovation.
And the fact that their system doesn't dump water. I think that is actually perhaps the bigger deal. Datacenters have been getting a lot of heat (pun intended) for using significant fresh water at the expense of local municipalities.
Closed-loop water cooling chips is nothing new. There are two separate water systems that often get conflated*. The loop warms up the water, which is recycled but first needs to be cooled externally somehow. Normally they use evaporative cooling towers for that, which do use water, or chillers, which don't use water but use more energy. But they're claiming they can get that water loop so much hotter than the outdoor environment that active cooling isn't needed. This could also be attributed to improving the closed-loop system to more directly cool the chips, which the article mentions.
Even air-cooled datacenters work somewhat the same way, but instead of water to chips, it's air. The air goes into hot aisles then exchanges heat with water, after which, see above.
* Other datacenter marketing materials talk about how they have a "closed loop system that uses no water" and they do still use water in the evap towers. I was half expecting this article to be that again, glad it wasn't.
The "innovation" is that everything is now attached to a watercooled block.
The rest is marketing: The Cray supercomputer were fluid cooled back in the 1980's, the entire board had an inert liquid flowing across it.
The innovation is being able to run the chips at higher temps without ruining them too quickly.
My partner lamented the same thing... Cray was doing this 40+ years ago
Cray used Fluorinert, a chlorofluorocarbon. So not exactly a environmentally friendly solution.
Bad quality of water clogging the pipes integrated onto the PCBs (thus requiring to replace the PCBs) was said to be what were killing those few USSR Elbrus supercomputer installations.
You have to design your hardware to tolerate being run in consistently hotter conditions. There's a tradeoff between cooling cost and failure rate / capex.
Nvidia's automotive and aerospace variants get ratings up to 85C, for comparison.
Don’t their consumer GPUs run at 85C core temp? Maybe not for as long though.
2 replies →
Is this not how it was already done? Huh.