Comment by swatcoder
2 days ago
Compare to the cost of professional-grade tools in other trades and craft hobbies.
Sure, $4000 can be a lot of if you're a casual hobbyist or are struggle to meet everyday lifestyle costs, but it's definitely not "insane" if this is the trade you make your living from or if you've established a lifestyle that affords disposable income for your hobbies.
And for some people, $4000 for a device you have complete control over and can repurpose and tinker with to your own needs and curiosities is a much much more justifiable expense than a $200/mo rental for some narrow-access tool that somebody else controls.
That's only half the reason it's expensive.
The other reason is that it would likely take years to spend $4000 (plus the real cost of electricity) worth of tokens on a 3rd-party provider that's running a similar limited, DS Flash type model. By that time, the hardware will be obsolete, assuming it's still operational.
> it would likely take years to spend $4000 (plus the real cost of electricity)
Since that cluster only yields 20-30 tok/s on that size of model, at least a decade before the hardware breaks-even with current token costs, and that's not counting electricity. Assuming continued downward pressure on token prices, and the cost of electricity, it never pays for itself.
I don't understand how people don't consider this.
Plus you're spec'd out of near-SOTA level in months.
The only reasons to actually do this are a) you have a lot of dispensable income and are a hobbyist/tinkerer, b) you have real, legitimate privacy concerns or, relatedly, c) you're doing something you don't want to get flagged
8 replies →
As a counterpoint, my homelab/home-LLM hardware has appreciated in value by about 60% since I bought it.
Of course, it's not real unless I sell, and the value will eventually go down, but so far I have significant paper profits.
Also, DeepSeek token prices are continuing to _increase_, not decrease.
7 replies →
> it never pays for itself.
Exactly; its a development box for fiddling with GPU hardware with a large amount of video-addressable memory. It's not an inference box, really, though it's neat that I can at all!
> The other reason is that it would likely take years to spend $4000 (plus the real cost of electricity) worth of tokens on a 3rd-party provider
That's just a one-dimensional thought! Your own hardware gives you complete control, and it doesn't time you out for 4 hours, unlike those vendors.
But if you can use cloud models, why wouldn’t you use SOTA? For 2400 USD or less per year you can get pretty huge amounts of benefit out of that (though at the whim of whoever you are giving the money to).
It's "insane" compared to the $1,500 it should have cost before the RAM crisis
I am pretty confident that given a $200 subscription on any of the big labs, you're getting $4000-$8000 per month in subsidized tokens... do what you wan't with your dough... and I too have a spark that I got really early (October 2025), but no, economically it does not compare to what's runnable locally in terms of quality from the frontier models. Economically, it looks like for as long as there are subscriber plans, you're better off renting.
Before getting the spark, I was just using a google colab account, their $49 dollar plan allows you access to h100's and I can run qwen there in a Jupyter notebook... and if I really need that web front end I can just use cloudeflair/tailscale/the local ssh client to reverse tunnel it.
This should be obvious but with a model running on local hardware you can do your own RLHF and mod its behavior however you see fit. With cloud hosted models you can't. A few years ago when the models were smaller there were people undoing the guardrails, censorship, and general lobotomization with some form of a RLHF training. You can't do that on larger models unless you have the hardware like this person does.
Notice all the comments saying like "omg why so expensive so just use the API??". It's a trick for lockin even with, so called, "open" models. Keep trying to run them locally, keep undoing the lobotomies, mod model behavior so that they work for you and do what you want vs only what someone else says they're allowed to do.
I love my Spark-like, but even for training you're better off using Vast or Runpod or whatever to rent cloud compute. Much faster and cheap as hell, to be honest.
I do set up my initial runs and likes like quantisation-aware-distillation on my Spark-like to test it out and get it working, so it has value! But its not "worth" it other than its fun hardware to tinker with, IMO.
> You can't do that on larger models unless you have the hardware like this person does.
Or just rent something substantial for like $4/hr on runpod or w/e to do that.
My gripe is this persons compute is wasteful and makes it harder for me to buy something with like 64gb ram to do normal work and run containers while I keep using cloud models.
Someone else calculated the break even being 10 years, it’s just dumb. And I think it’s clear there won’t be a big rug pull anymore, there are too many open models and providers now.
Anecdotally, ~$500-1500/month token spend at API OpenAI/Anthropic pricing seems pretty realistic for full-time engineers at companies with "liberal but not unlimited" LLM spend policies.
This is of course anecdata. I know plenty of outliers, too. I know a principal engineer who uses many multiples of the number I quoted above. I am sure we also know many people making do with much much smaller budgets as well, via all kinds of well-discussed methods.
But, "$500-$1500 per month per full-time developer" is just kind of the personal mental baseline I use when making my decisions with regards to thinking about whether any of this makes any economic sense.
With multiple 200 a month subs you are getting a multiple of those subsidized tokens. At least if you tabulate at retail api prices.
This rent in the era of expensive hardware thing is not exclusive to inference.
I’ve needed x86 architecture for windows builds recently and have just hemmed and hawed over buying a decent windows 11 box.
I can’t make the math work against Azure instances.
I can spin up a nice one for build deallocate,spin up something cheaper for QA and then turn that off.
I can build all the devops around that, with a number of passes, with a skills based interface so working with the cloud is not too bad.
The only thing that still has me thinking about it is the prospect of price is going up even more, which is acid as far as I know.
And I’m hopefully going to need this x86 stuff enough that I don’t wanna wish I had gotten one for that high prices now.
The cloud stuff is definitely a much better economic value, but I would argue:
1. You learn a lot more running this stuff yourself (especially since you can poke at its internals if you're interested or watch the reasoning chain.) Just being a consumer of this stuff doesn't really teach you much about it other than model & harness specific tricks that become obsolete pretty quickly. (IE, your Claude.md from 6 months ago probably needs a rewrite). Which is fine, I don't think you're going to be "left behind" if you're not a hardcore AI enthusiast or anything (I'm not), but as a guy that's always been interested in computer science I want to see how it ticks.
2. You can't really depend on this subsidization lasting forever IMO. I know the financials thing has been beaten to death but I guess I'm in the camp that it's good to be in control of your tools so that you can go elsewhere if the economics change.
I like to check in with ccusage pretty frequently, and honestly like if I were paying API prices for Claude I'd probably be paying thousands a month.
3.privacy
Any organisation or individuals not wanting to have their sensitive data flowing away (either because of trade secret or data protection laws)
1 reply →
I am also not sure I would choose to use the cheap and easy to run at home model, given a choice. The marketing copy says this is a frontier model, but it's not. Sol and Mythos are the frontier right now. GLM 5.3 Flash simply isn't. I'd rather use the frontier model as they waste less of my time than even Opus.
Yeah, in any other profession where you need to buy a van to drive stuff around, you easily spend similar amount of money on capital investment.
For $200/mo you either have a SotA model you can’t run on those devices or you have a cheaper model where you pay less than $200 or have a really big amount of tokens without the energy costs and the risk of failing machine