Comment by lightbendover
7 hours ago
Who is buying these at 70% of new pricing given the sky high likelihood of them being shot? Maybe it's safe to buy from small labs that went under quickly, but I can't imagine a cluster that has been operating near its thermal limits for a couple years fetching that kind of resale.
Why would they be shot? Unlike the consumer cards that are basically factory overclocked to look good on benchmarks, the datacenter GPUs are designed to run at full tilt 24/7 and survive for years.
According to the article these cards have a 9% annual failure rate.
>The number traces to Meta’s Llama 3 technical report, which documented 419 unforeseen disruptions across 16,384 H100s over 54 days of training, of which 148 were GPU failures and 72 were HBM3 memory failures.
From an annualized number on the llama 3 training report. would be interesting to see if we have a better idea given that we're already on rubin.
Most hardware has an exponential (memoryless) failure distribution in practice. The bathtub curve is a myth.
It depends entirely on the time-to-failure distribution though whether used cards are a good deal or not. Often this kind of hardware has a bathtub shaped hazard rate, actually getting burned in cards may mean you get the weeded out solid specimens, and forgo the lemons.
Given the decades of consumer and enterprise GPUs often being identical or near identical hardware, it'd be interesting to see if there's any evidence of this actually being true.
It was true of crypto GPUs too, although mostly people picking them up for gaming. Always seems high to me too but if you can get any guarantee of them not being on fire when they were pulled the bathtub curve keeps you pretty safe, thermal limits are limits for a reason.
crypto gpus didnt run at 100% power draw so the strain wasn't that massive
Why weren't crypto gpus running at 100%?
5 replies →
I bought a RTX 3070 off a miner when Eth went to proof of stake.
It was clean, cheap, and is still going strong for daily gaming.
In its working life it was undervolted and probably cooled better than in my rig.
1 reply →
I feel like this is not as important as people make it out to be.