Comment by SwellJoe
12 hours ago
I have a Strix Halo and dual 32GB GPUs in my desktop, that sit idle right now, because the electricity to run them and to cool them in 110F weather Texas is currently experiencing pretty much nulls any savings I might see over getting better models from cloud providers. While I mostly use Claude or Codex with subscriptions for agentic work, for API use DeepSeek has usually been my go to, but now I guess it's GLM 5.3 or the Flash version. And, for security work that Anthropic or OpenAI models are likely to refuse, I've been using Kimi K3 (also via subscription, though their subscription is extremely stingy), but I guess GLM is now the one for that, too.
Anyway, yeah, even at the prices I spent on my local AI stuff (I bought before RAMpocalypse really kicked into gear, so I bought old server GPUs for about $350 each and the Strix Halo for a little over $2k) it was never going to pay for itself; I just like to tinker. But, I can't imagine spending today's prices for hardware for local AI.
When the memory shortage ends, I'll be down to the Apple Store (or, more likely, clicking refresh on the Apple outlet every few days). But, until then, there continues to be a glut of cheap and free models in the cloud that are better than anything I can run locally and they're faster, too.
I wish Texas would write up a regulation allowing 'balcony solar' as I could easily generate 1000-2000w of solar in my small back yard to take a bite out the sizeable cooling bill I have.
Seems like it's easier to ask forgiveness than permission. And, I wouldn't bet on this legislature ever doing anything that would disempower fossil energy or reduce their profits, even a little bit.
Too hot and expensive to run right now but a great hedge for peace of mind against $200 subscriptions shooting up to the $4000* they should cost.
*$1000? $14,000? Who knows but everything in the middle there has been claimed.
Yeah, I guess, but it feels like there isn't really an opportunity for anyone to do that, given how competitive the market is. If Anthropic decides to demand API rates for everything (which would make my $100/month turn into a few thousand, I guess), I won't be seriously inconvenienced by switching to GPT. And, if both of the major American providers do a pricing collusion and GPT also becomes thousands of dollars a month to use, I can choose between Kimi K3 and GLM and so on. I'd rather use Opus 5 and Fable, but I'm not going to be seriously put out if I can't. We've got three or four open models to choose from that are as good as or better than Opus 4.8, which is Good Enough, and the competition isn't slowing down. We're seeing more new competitive models more frequently than even three months ago.
So, even though there are more models to run locally that can be useful for the stuff I do, it makes less sense now to do so than it did when I got it. There are more extremely cheap options, now, and it seems likely to continue to get cheaper and better and faster, while my local hardware will always be slow and hot and only gets better via software (which has roughly doubled the speed it can run stuff since I got it, but it seems like there's less room for improvement in software now, and even at twice the speed, it still kinda sucks to use local models interactively especially on the Strix Halo).
If they "should" cost 4k in the sense of marginal cost, then you will be spending more running the same at home, because your home hardware will always be less efficient.
There is a big difference in the cost of a 5-nines up time system in a heavily space constrained environment compared to a home hobby white box used for some coding. The GPUs alone cost 10x for the data center versions compared to the gaming versions even with similar specs.
The cost of online services is also largely a result of the cost of training (though hard to say exactly what that number is). Assuming you are using open weight models at home, you aren't paying for the training - someone else is.
1 reply →