Comment by bastawhiz
5 days ago
Whether you think building data centers or not is a good idea it's inarguable that the per-token efficiency (power, hardware, etc) is FAR higher in a data center. That's literally what it's designed for.
5 days ago
Whether you think building data centers or not is a good idea it's inarguable that the per-token efficiency (power, hardware, etc) is FAR higher in a data center. That's literally what it's designed for.
im talking per value. look at the efgiency of chinese open source models; then look at SOTA sucking gigawatts, then the proposals.
America is basically proposing AI using the equivalent bloatware of Windows 11.
I run two 49B parameter models on a pair of used A100s full time and it sucks down 250 watts at peak utilization. That's not gigawatts, and it's completely within the realm of comparison to what the author is describing.