Comment by UncleOxidant
1 day ago
Sure, this is a small anecdotal datapoint, but since I've gotten Qwen3.8-Flash-Next running well on my Strix Halo box my Claude Pro subscription use has dropped off a cliff. I'm only at 18% of quota usage this week - this being Friday I would've been at 80% at least in the past. This is just an astonishingly good model that runs locally and quite well on my PC. I do not plan to renew my Claude Pro subscription when it comes up in a couple of months.
I really hope the future is local models, because the idea that all companies will need to buy tokens from a handfull of tech companies to run their businesses or make their products is awful.
I've been ripped (on this site) for making this prediction. While the frontier (i.e. The toughest problems) will always push the limits of available computing power, what typical people do with AI won't always be on the frontier, nor will what appears on the frontier necessarily be lucrative. The frontier used to be playing Chess, Atari, and Go. These were challenging and really advanced AI, but they didn't make a lot of money. Now a phone can do these things.
You're an example of this process. You used to require a frontier model running on a data centre. Algorithms improved and, perhaps, your hardware as well, but your requirements didn't go up. Now you can do what you need to do with local hardware. You're one AI user who might not need to send money to Anthropic going forward unless you find a new frontier application that interests you.
This is why I'm a little bearish when it comes to the valuation of the AI giants. Yes, there are some extremely lucrative things on the frontier right now but, in the future, people may shift a lot of what they're doing now to local hardware while new things on the frontier may not produce the kind of profits that will pay back trillions in debt. AI companies have bet big on themselves being able to constantly and perpetually find new applications, and also that those new applications will make money.
I agree with you that if the trends of 2025-2026 continue into 2027-2028 for open weight models and local inference, 120b param open models will be more than good enough for 80% of today's use cases. If so how can you justify OpenAI and Anthropic and the AI buildout?
> but your requirements didn't go up.
I am definitely building more and more ambitious projects as models improve. Both personally and at work
1 reply →
I think we will see good enough local models, excellent cloud models, and then simpler focused models embedded on silicon running on every small device.
Why only a handfull? There can be (maybe already are) hundreds of LLM API providers.
There’s no way to make that economic.
3 replies →
> the idea that all companies will need to buy tokens from a handfull of tech companies to run their businesses or make their products is awful.
To the leadership of these tech companies that actually is the dream. Sam Altman said the quiet part aloud recently: "From a global allocation perspective, actually, it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer, and being used maybe 5% of the time."
And it certainly fits into the neofeudal worldview of other relevant Big Tech leaders. Be it Peter "I no longer believe that freedom and democracy are compatible" Thiel, or unnamed bigwigs discussing if shock collars are effective to prevent their staff from revolting after a doomsday scenario [2].
We shall own nothing and rent everything, that is the mindset these people believe.
(And if you ask me: the AI IPOs are part of that strategy. Assuming OpenAI and Anthropic both IPO, they will get part of every American's and most other countries' pension funds... and when they collapse, a lot of that value will be gone. The initial investors into SpaceX, OAI and Anthropic all will have made their money and put it aside, and we will be left holding bags)
[1] https://www.ign.com/articles/openai-ceo-sam-altman-thinks-it...
[2] https://www.rte.ie/news/business/2025/0511/1512086-doomsday-...
How is that saying the quiet part out loud?
2 replies →
Altman was simply making an accurate observation about DRAM, in the context of a conversation about DRAM shortages and precisely why it is difficult for DRAM manufacturers to respond to the market price incentive to increase DRAM production. Even if you think it is bad that DRAM is currently expensive, as I do, that doesn't change the underlying fact about the world that Altman's comment reflects.
Thiel's comment is also pro-individual-ownership - in context he was talking about the danger that in a mass democracy the people as a whole would vote to have the government restrict the freedom and confiscate the property of successful people (particularly prescient in light of the recent California attempt at a billionaire wealth tax). RAM being expensive is better than RAM being rationed by a democratically-elected government so no one person has too much, as judged by the voting masses.
4 replies →
> the idea that all companies will need to buy tokens from a handfull of tech companies to run their businesses or make their products
Did you somehow become incapable of doing things without models? Most of us have not.
This is exactly what I'm seeing all over the place. I'd like to say it's not true for me, but I'd probably be lying a little bit to say so. It is causing some atrophy in my skillset. One could argue that I'm gaining something different through LLM usage like "getting good at asking the model questions" or something, but it doesn't frankly feel like a skill which is all that useful on a going forward basis.
Imagine if only KB Home and D.R. Horton had access to power tools while all the other homebuilders big and small had to use hand saws and screwdrivers. Would they be incapable of building houses without power tools? No. Will they be relatively screwed as companies? Yes.
1 reply →
Interesting, I'm also comparing 3.8-flash-next (4-bit quant, llama for inference & pi as harness, on a 128gb strix halo machine) with opus 5.5 in claude code and while qwen is a good and fast model, it's nowhere near for coding, even haiku 5.5 feels better.
I could see it used as a sort of "sub-haiku" for easy stuff to leave more tokens for the bigger claude models, but I won't be cancelling my Claude sub any time soon.
It's obviously nowhere near opus 5.5, but it's crossed the same threshold opus 4.5 did: it's good enough. If all model-as-a-service companies shut down tomorrow, I'd be happy with qwen 3.8 flash next q4 running in my strix halo
The problem here is we're not talking ... I don't know ... hamburgers. If I like McDonald's better than Burger King, it's not going to stop me from buying a Burger King burger (if I'm hungry and there's no McDonald's around). One burger may be a bit better, but I can reasonably substitute one for the other.
Coding AIs aren't burgers. Even if a local LLM is as good as Opus 4.5 (when you sink several thousand dollars into hardware ... and I'm skeptical even then) ... the actual difference between the models is huge.
Let's say I use the $200/month Claude (I don't, I use a cheaper plan, but stay with me). If I switch to a Opus 4.5 equivalent local LLM, and it makes even a single production issue that 5.5 wouldn't, that mistake will cost thousands of dollars.
That "cheaper" model isn't really cheaper.
9 replies →
> it's crossed the same threshold opus 4.5 did: it's good enough
Agreed on that, and actually I'd put it somewhere around opus 4.5 from what I've seen so far.
>> It's obviously nowhere near opus 5.5, but it's crossed the same threshold opus 4.5 did: it's good enough.
Good enough may be sufficient for personal projects, but in a competitive landscape where you're using a "good enough" model and your rivals are using a frontier model like Opus 5.5, you will be at a significant disadvantage.
2 replies →
> If all model-as-a-service companies shut down tomorrow
Is this the nerd version of "doomsday prepping"?
4 replies →
Here's a stress test of sorts that I've had it working on the last couple of days: We have a C -> microcode compiler written in C (most of that was work done 'back' in the Opus 4.6/4.8 era). I gave it the docs for that compiler and asked it to implement it in OCaml (since it's not very common, but also is a good implementation language for compilers). It can use the first compiler as a golden to compare against. It's essentially: C language parser -> AST -> IR -> [optimization passes] -> microcode (+ some verilog code). It's basically at the point now where the original compiler has been. Not only does this generate microcode it also has to generate some Verilog code that loads and runs the microcode as well as run simulations in Verilator to confirm functionality. I asked it to come up with some new optimizations that the original compiler doesn't have and it came up with a couple that in some cases reduce microcode lines by up to 30% (less microcode in this machine means more performant). These are ideas that Claude didn't come up with when it was working on the original compiler back in the 4.6/4.8 era(and I recall prompting it several times asking it if there might be other optimizations we could perform).
So all in all, I put it at Opus 4.8 level, at least. And in some ways it's exceeding 4.8.
I used Ollama with Qwen 3.8 27b on my MBP for a flight to try some experimentation and found that it was shockingly good at one of the more complex PDF processing tasks.
Like seat-of-the-pants good as Opus 5.5 but a bit slower and required just a bit more prompting to get what I wanted (which is fine for an app pipeline).
And only slower because hardware
I am in the same boat. Sun servers were still the best that money could buy when they lost to the Linux revolution. Linux was _good enough_ and destroyed their margins.
I would have preferred to have open Solaris flourish along with Linux.
That jives with my own datapoints. Of a circle of around 20 very active devs Anthropic went from being the must-have tool to on the chopping block.
Around 50% of that group says they plan to cancel their Claude subscriptions now that their local or open model setup is performing “good enough.”
I don't believe this one bit. There is no way local models are good enough except for the most basic of coding.
If you actually know the code and can give guidance, direction, and corrections they absolutely are.
If you are just vibing and managing multiple agents and reviewing at a higher level like architecture, you are right.
Don’t know what to tell you. It’s happening.
Why is it unbelievable? I still remember how to program and arch new features and don't need claude to tell me how to do it. It is much more enjoyable to do a little work on your project and offload the grunt work to a weaker model then just being a highly paid merge pr button pusher for whatever opus decides.
1 reply →
Local models: if they have the hardware to run it.
Open models: we're already seeing this at some scale. https://news.ycombinator.com/item?id=50350004. I do expect to see more of this.
What do you consider a local model? I'm running DeepSeek Flash version 4.1 and it's outstanding.
You haven't spent much time with the latest self hosted models then. Give it a good harness and instructions, access to a web search engine and ability to iterate on problems and they do a good enough job that I'm happy with the results.
I hate to break it to you but most people do most people do basic things most of the time
[dead]
These companies have no moat at all.
Their moat is their research capability. It's a combination of proprietary internal technology and models and infrastructure, people, organizational structure, and culture. As long as they maintain the best and fastest research capability, they have a durable advantage, at least for the time being. If we start to saturate all economically useful outcomes that can be reached via greater research productivity attainable to us, then this moat will start to fall apart. It could also fall apart if safety issues force a premature halt or major slowdown to further research.
This assumes any research gains last. In practice it seems like none of the frontier companies can keep much of their "secret sauce" secret for very long.
If company X has a better model than company Y, do you use it? Well, that depends: will Y take two years to catch up? If so yeah, you buy X.
But if Y is just as good a few months later, you purchase based on other factors, because you know you can live without the brilliant research X is doing ... since you'll have it soon enough.
The problem with your logic is that the history of tech is full of “good enough” eating “best” for lunch all day long. Anthropic’s plan to be the best is increasingly looking like a terrible business plan.
> Their moat is their research capability.
That doesn't seem like much of a moat. What happens when a couple researchers go on new parent leave at the same time, or someone's productivity drops for a quarter because a parent passed, or gets divorced, or whatever?
Continually staying 6 months ahead of the distilled open-weight models means you need to keep flipping heads (nobody goes on vacation) until the end of time. The geometric distribution describes that, and it ain't pretty for the expectation value of these companies.
How long till that "research capability" becomes just 5 people with many AI agents, and no other humans in the main research loop?
3 replies →
> Their moat is their research capability.
Research does not bring in revenue. Products or services do.
3 replies →
You mean, if they keep running quick enough they are save? That's like the compete opposite of a moat, in my mind at least.
That's not that big of a moat though, as we've seen with DeepSeek and even smaller teams doing edge model work.
1 reply →
I’m not sure I’d call maintaining the fastest research program a “durable advantage”.
If their moat is their ability to create marginal improvements every few months by releasing new models, then that is an extremely expensive moat to maintain.
I think the staggering amounts of compute and capital at their disposal is a kind of moat.
You need those staggering amounts of compute to run an inference service, but you don't need it to run a powerful model.
It doesn't cost a company that much to buy a box capable of running the Qwen3.8-Flash-Next model in the GP comment and never pay Anthropic again. The cost will be even more reasonable once the supply issue is solved for that hardware.
AI-as-a-service is a unique product because when the interface is natural language, there is no lock in. VS code has a hotkey to swap between claude, openai and a local model. Compare that to the difficulty of swapping from macOS to Windows.
1 reply →
Sure, but anyone with money could do that.
We're the only people that can spend this money is not a moat.
I wouldn't be so sure about capital in future. I think there might be real look for return on any new capital they are getting. Be it equity or debt. And that alone will not be great for them.
How long they will have access to staggering amounts of compute is uncertain and not guaranteed, especially if they get a junk or near-junk credit status when they go public.
The capital is running out and the compute is a massive liability on their balance sheet. Your definition of a moat is rather odd.
How much of their compute is leased/rented? Last I read it was quite a lot of it.
the idea of a moat is that it's an inexpensive way to keep the invaders at bay.
If you need to buy staggering amounts of compute, spend staggering amounts in utility costs and lobbying, and keep spending that as your hardware amortizes and your operations go on, what you have isn't a moat, it's a standing army. It's extremely expensive, and the day you start having issues, mercenaries will cut your throat before you know it.
That argument seemed to have legs in 2022 when OpenAI was the only game in town.
But the fact that were even talking about Anthropic like this only a few short years later demonstrates that that was only kind of a kind of moat.
The current compute they are buying is NOT state of the art (check this year's academic research on inference hardware). There is 10x-100x inference-per-watt improvements in existing prototypes (in the industry, not just academia). Not sure what I am talking about? Ask your friendly AI agent of choice, for the details.
Check out mythic.ai for one example.
compute and debt*
Banks are going to want that "capital" to materialize into gains at some point. The water in that moat is evaporating.
Capital only means anything if you can spend it to buy resources like compute and turn those resources into additional value.
Hardware's ridiculously expensive. Then, there's no good place to put it, because the new breed of tech bro managed to propose the data center buildout in the most exploitative, alienating, and electorate-angering way possible. So compute's hard to buy.
The moat is the data they have collected. There is no way that the "de-identified" data from enterprise customers isn't making it into the training set one way or another.
Even worse, their "moat" depends on buying the most overpriced hardware in huge quantities and subsequently eat a huge amortization loss in a few years when it's obsolete.
I’m waiting for somebody to use one of the frontier models to hack its own data centre and exfiltrate everything…
it feels like this is a bit similar to the famous dropbox comment.
I guess maybe another anecdotal comment, but I'm an engineer myself, have a home server for simple stuff, and I pay $200 a month to chatGPT for a pretty simple usage and really don't see myself go and run my own open source model, especially at the speed at which they evolve and change.
99% of the world will not setup their own open source model, just like 99% of the world is not setting up their own servier with ssh copy whatever the dropbox comment was saying.
You don't have to rub it yourself though. There's lots of pure inference providers which are way cheaper than the big guys and have cheaper models
If you were paying API prices (for ZDR), I suspect you would be eyeing local hosting. But if you’re comfortable selling your data, then the subscription prices are going to be hard to beat, for however long they may last.
99% of the world is also not spending $200/month for chatGPT. They're using whatever free shit they can get to ask basic questions.
Right. And FWIW those people are actually quite close to being able to do what they need with their phones.
If it were not for the x86 architecture they would almost have models that could answer those queries on their desks. (Gemma 4 E4B QAT is surprisingly good at this stuff; Gemma 4 12B QAT is great at it).
If I was Sam Altman I would consider giving Intel and AMD money to not develop unified memory architectures.
I’m migrating that way as well, but I’m still struggling in creating a nicely tuned harness to achieve what I really want:
Opus for planning and the very difficult parts, Qwen for the mundane ones. It still involves a lot of manual model switching (but opencode and Pi make that relatively easy).
Curious how much ram you need to run this at acceptable performance
I've got a prototype cluster of 5 devices running Qwen3.8-27B assigned different roles and sharing outputs between each other. Beats Claude Opus MAX in SWE and Terminal benchmarks.
This. The problem is now the same monopoly cartel problem.
Strix halo is double the cost because of...
How much does such a box cost?
About 4000 eur
And 4 is about to drop.
Will be interesting to see how much of an advance 4 will be. I was under the impression that 3.8-flash-next is basically a 4-preview. At least architecturally, but as we're seeing nowadays (with DeepSeek, Kimi 3.1) "continued post training" can add a quite a bit.
4 is a generational leap compared to 3.8 flash.
2 replies →
There's basically no point in paying for these subscriptions anymore. Sure they are marginally better at some things. But they are also marginally worse at others and the implications suck. I stopped all my subscriptions and keep like 20 bucks in open router just in case. Never need it, but it's there...
The non-anecdotal part is to me as interesting as the anecdotal part. The price of RAM has gone up a great deal which is going to make it harder for others to get this sweet setup that you have.
I see lots of people talking about running Qwen as a local model just curious what's the difference with say something like local deepseek?
Qwen 3.8 Flash Next (125B, 6B active) is minuscule compared to DeepSeek V4.1 Flash (552B, 16B active) and GLM-5.3 (753B, 40B active). It's more difficult to run the latter 2 locally at usable speeds.
The latest Deepseek Flash is much, much larger than Qwen3.8 Flash next
Qwen-3.8-flash-next hardware costs motorcycle level money, DeepSeek hardware starts at mid range car money.
How does it compare to haiku?
I guess it depends on which version of haiku? I really think that right now this is basically Opus 4.8 that I can run at home. In some ways possibly even better.
What inference engine do you use ?
Gufo. It's the best open source inference engine that's optimized for Strix Halo right now. Halogen may unlock a bit more performance, but it's not open source and gufo seems to be making steady progress.
Cool. I found halogen but was turned off by the closed source nature (github repo that just turns out to run a download shell script, lame).
Do you mind sharing more details about your setup? I have been using opencode with lemonade for inference on my strix halo but haven't quite hit the sweet spot.
1 reply →
Do you ever worry whether these open source models have been trained to embed backdoors or leak secrets in very specific circumstances at some point in the future? The kind of thing that's hard to know ahead of time and even detect after it's happened?
I feel like the industry hasn't solved that alignment problem yet... We have no way to verify the entire models landscape of intent.
Paid models have an incentive not to do that because if the market discovers it there is a huge economy penalty, but the free open source models don't have that disincentive.
> Paid models have an incentive not to do that because if the market discovers it there is a huge economy penalty, but the free open source models don't have that disincentive.
False.
I worry more about Claude.
https://news.ycombinator.com/item?id=50387628
> False
What exactly are you claiming is false about my statement?
The list you presented are mistakes Anthropic made or cases when they put their own interests ahead of users, but I don't see how it disproves the point I was making. None of those are examples of embedding backdoors or stealing secrets to take over or sabotage a system. They are categorically different than what I'm asking about.
Where's the huge economic penalty?
> Paid models have an incentive not to do that because if the market discovers it there is a huge economy penalty, but the free open source models don't have that disincentive.
This just fundamentally isn't true. They run on the same economic model, but they have to release the weights to gain credibility, unlike the American labs.
> Do you ever worry whether these open source models have been trained to embed backdoors or leak secrets in very specific circumstances at some point in the future?
Open weight models are extensively analyzed. It is extremely unlikely there is an intentional 'backdoor when X' concept added to any of them. Not impossible, but unlikely for the same economic incentives you see in the other labs, and given that no evidence of anything along those lines has surfaced yet.
We've seen many open source software libraries, frameworks, and hosting platforms with vulnerabilities, backdoors, or leaks.
In my opinion, to suggest open-weight models are extensively analyzed while being net new and continuously released and the model itself being unintelligible to humans seems a bit premature.
It's possible that closed-weight models end up following the pattern of being arguably less secure since there are fewer eyes on them analogous to open source software but I don't think there's a perfect match here.
Should be soon solved with open weight model providers offering enterprise support.
No, I'm not much worried. I think this is mostly FUD at this point to try to get people to not run local models and keep paying the "trusted" providers. But paid models have been exfiltrating data - there have been several stories about these incidents here.
"trusted providers" don't even need a backdoor, they just collect everything from you and whether you approved or not they're using that info to train the next model.
I feel that way but about Anthropic and OpenA lol. Didn't they just get caught sharing your email with every website every time it does a search?
Backdoors in model weights are mostly theoretical. And the "alignment problem" that the AI labs love to talk about really doesn't exist either, at least, not the way they use it. The real alignment problem is getting an LLM, which is just math, to obey arbitrary human rules. In other words, LLMs do not have "alignment" at all. However, AI CEOs abuse the term and use it to mean "AI has an alignment, just not the one we want".
I don't ship AI-written code without at least having an unrelated model of equal capability review it. That should cover a multitude of sins like intentionally-embedded backdoors and zero-days...
... as long as the reviewer model hasn't been deliberately lobotomized, that is.
(I don't disagree with the [dead] comment, either -- fortunately I tend to make small changes to code I'm very familiar with, so it's fairly easy to vet in person. The more code gets generated by the clanker, the more nervous I get, just because it's impossible to keep some complacency from creeping in.)
I don’t think parent was talking about the shipped code, but about your one development environment while using the models.
1 reply →
insane. I can't imagine doing this and I hate that I'm being forced by upstream communities to run code nobody has read
just read the damn code yourself!
2 replies →
[dead]