Comment by elorant
7 days ago
There have been discussions about this chip here in the past. Maybe not that particular one but previous versions of it. The whole server if I remember correctly eats some 20KWs of power.
7 days ago
There have been discussions about this chip here in the past. Maybe not that particular one but previous versions of it. The whole server if I remember correctly eats some 20KWs of power.
A first-gen Oxide Computer rack puts out max 15 kW of power, and they manage to do that with air cooling. The liquid-cooled AI racks being used today for training and inference workloads almost certainly have far higher power output than that.
(Bringing liquid cooling to the racks likely has to be one of the biggest challenges with this whole new HPC/AI datacenter infrastructure, so the fact that an aircooled rack can just sit in mostly any ordinary facility is a non-trivial advantage.)
> The liquid-cooled AI racks being used today for training and inference workloads almost certainly have far higher power output than that.
75kW is a sane "default baseline" and you can find plenty of deployments at 130kW.
There's talk of pushing to 240kW and beyond...
> Bringing liquid cooling to the racks likely has to be one of the biggest challenges with this whole new HPC/AI
Are you sure about that? HPC has had full rack liquid cooling for a long time now.
The primary challenge with the current generation is the unusual increase of power density in racks. This necessitates upgrades in capacity, notably getting 10-20 kWh of heat away from few Us is generally though but if done can increase density.
HPC is also not a normal data center but also usually doesn't have the scale of hyperscaler AI data centers either.
Well for some. Google has been using liquid cooling to racks for decades.
That’s wild. That’s like running 15 indoor heaters at the same time.
20KW? Wow. That's a lot of power. Is that figure per hour?
What do you mean by "per hour"?
Watt is a measure of power, that is a rate: Joule/second, [energy/time]
> The watt (symbol: W) is the unit of power or radiant flux in the International System of Units (SI), equal to 1 joule per second or 1 kg⋅m2⋅s−3.[1][2][3] It is used to quantify the rate of energy transfer.
https://en.wikipedia.org/wiki/Watt
If you run it for an hour, yes.
Ah yes, like those EV chargers that are rated at X kWh/hour.
1 reply →
I asked because that's the average power consumption of an average household in the US per day. So, if that figure is per hour, that's equivalent to one household worth of power consumption per hour...which is a lot.
9 replies →
It’s 20kW for as long as you can afford the power bill
20 kWh per hour