Comment by __float

Yes, the tunnel vision around local models on this site is crazy. The percentage of people in the world who can afford the hardware is extremely low.

I keep coming back to this: why do I need to run a local model on my own GPU? Open models can run in dedicated clouds and while, yeah, they may be more expensive per token than my own GPU, when accounting for depreciation, energy usage, and opportunity cost (money not spent on my GPU will instead sit in my portfolio appreciating with its particular blend of returns), I'm pretty sure I break even or even net lose money with a GPU.

Don't get me wrong, there are advantages to a fully local model in that, I can have agents looping 24/7 even when my internet is not working. But this is niche enough that if I had to price the advantages they don't seem worth it.

If I'm willing to pay the Openrouter tax, I can fire up Openrouter today and just get access to whatever model I want, and still pay a fraction for tokens as what I'm paying with the big guys.

My single 3080 runs so hot I don't need to warm my room in winter, and have to play games in my underwear in summer.

Unless you value privacy, pay for openrouter. You still get the benefits of cheap tokens and programmatic usage.

3090 pricing is something of a wild card. Since the only big-mem consume cards are the xx90s, and a 5090 is pushing $5000, resale value has gone way up. The bottom hit ~$700 last year. It's still a very good GPU, if power hungry.

Most people in the US have a car, and the average new car is $40,000. Hell where I live a middle class consumer will spend double that on a Boat or an RV and think nothing of it. These aren’t elite tech workers.

It’s not unfathomable that if a personal, generally intelligent local AI provides enough utility and doesn’t require you to tweak CLI flags millions of Americans would want one.

where? i would love that.

They cost more to run than hosted anyway. But that isn't the point of having them. They are a playground, a backup when the internet is down, or claude is down. They can render Blender scenes pretty well. They play any game I want.

You can do each of those at various hosts and own nothing. Or own a couple "over priced" cards and do it all at home on battery power for a few hours while the power is out.

But after all those years you’d still have 2 3090s, which are now about 6 years old and still holding value.

I keep seeing this comment. This is _hacker news_ where, back in the day, people just hacked on things, because it was a hobby. They weren't "moneymaxxing" or desperately trying to be as insanely efficient as possible. They hacked on stuff with a can of surge at 3am because it was fun.

Your comment is like a meta comment of "LLMs are generating everything, after a while the ouroboros will eat itself. (Which I agree with)" If people aren't hacking on this shit just because, you have completely conceded control of software to a handful of sociopaths, and open source software is dead.