Comment by somenameforme
2 days ago
And anyone with 3 can also see another trend. The hardware required for inference is rapidly falling. It's all following what happened with text-to-image image gen stuff almost to a t. It started out requiring super computing clusters, and now a days you can do high performance image gen on any plain old computer. I think within a few years we're going to be running frontier level models on a basic mid-range gaming hardware. You can already run quantized frontier level models on $1600 tier cards at high performance.
Seeing this bubble play out in real time is the most surreal thing. It's like watching a train wreck play out at a speed of a few inches per day. Like come on, just get off the track FFS.
Yes and this year is the year of linux on desktop
Except the plummet has already happened and continues to happen. As mentioned, you can run quantized frontier tier models on cards with a $1600 MSRP. So your comment would be like stating that this year is the year of Linux on desktop if Linux had already gone from 0.01% to 20% marketshare in a few years. At which point that would indeed be quite the reasonable prediction!
You mean the cards with $1,600 MSRP that regularly sell for $7,000+ on eBay?
1 reply →