Comment by mbrumlow
18 hours ago
It’s not that they don’t work. It’s how businesses handle hardware.
I worked at a few data centers on and off in my career. I got lots of hardware for free or on the cheap simply because the hardware was considered “EOL” after about 3 years, often when support contracts with the vendor ends.
There are a few things to consider.
Hardware that ages produce more errors, and those errors cost, one way or another.
Rack space is limited. A perfectly fine machine that consumes 2x the power for half the output cost. It’s cheaper to upgrade a perfectly fine working system simply because it performs better per watt in the same space.
Lastly. There are tax implications in buying new hardware that can often favor replacement.
I’ll be so happy to buy a EOL H100!
But no, there’s none to be found, it is a 4 year, two generations old machine at this point and you can’t buy one used at a rate cheaper than new.
Well demand is so high currently that it's likely this cycle doesn't exist yet for fast cards.
For servers I've seen where the slightly used equipment is sold in bulk to a bidder and they may have a single large client buy all of it.
Then around the time the second cycle comes around it's split up in lots and a bunch ends up at places like ebay
Yea looking at 60 day moving average on computeprices.com H100 have actually gone UP in cost recently, at least to rent.
A lot of demand out there for sure.
Not sure why this "GPUs obsolete after 3 years" gets thrown around all the time. Sounds completely nonsensical.
Especially since AWS still have p4 instances that are 6 years old A100s. Clearly even for hyperscalers these have a useful life longer than 3 years.
I agree that there is hyperbole thrown around a lot here and its possible to still use some hardware for a long time or to sell it and recover some cost but my experience in planning compute at large companies is that spending money on hardware and upgrading can often result in saving money long term.
Even assuming your compute demands stay fixed, its possible that a future generation of accelerator will be sufficiently more power/cooling efficient for your workload that it is a positive return on investment to upgrade, more so when you take into account you can start depreciating them again.
If your compute demands aren't fixed you have to work around limited floor space/electricity/cooling capacity/network capacity/backup generators/etc and so moving to the next generation is required to meet demand without extremely expensive (and often slow) infrastructure projects.
1 reply →
It's because they run 24/7 in a challenging environment. They will start dying at some point and if you aren't replacing them you will have a big problem when they all die en masse at the same time.
These things are like cars, they don't last forever and break down with usage. Yes, they can last 7 years in your home computer when you run it 1% of the time. They won't last that long in a data center where they are running 90% of the time.
3 replies →
There’s plenty on eBay? But at the end of your comment you say “a rate cheaper than new” so maybe you mean you’d love to buy a discounted one. But they do seem to be available used.
> so maybe you mean you’d love to buy a discounted one
Yes. I'd expect 4 year old hardware used constantly in a datacenter to cost less than when it was new!
(And just in case you did not look carefully, most of the ebay listings are scams. The actual product pictured in those are A100 workstation GPUs.)
> Rack space is limited.
Rack space and power (and cooling) in the datacenter drives what hardware stays in the datacenter
Do you know how support contract lengths are determined? Seems like a path to force hardware refreshes with boilerplate failure data carried over from who knows when.