Comment by GreenWatermelon
17 hours ago
Until the Chinese stuff came online I was completely convinced this is just a bubble waiting to pop and nothing will come of this tech due to its absurd costs. The Chinese stuff changed this picture massively. I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy + I need to follow along. Opus 5.5 speak is still cause aneurysms for my brain. It also has this vibe to its prose... like it's looking down on me. This is Prolly my bias speaking though.
> My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy
For such workflows it barely matters which of a dozen models gets used.
Whereas if you're handing off all dev and no longer reading code, just making CTO-level architecture decisions, then there are only 2~3 models right now worth using, none cheap.
> Whereas if you're handing off all dev and no longer reading code, just making CTO-level architecture decisions, then there are only 2~3 models right now worth using, none cheap.
I use frontier models every day and night, but not sure if the described approach is possible with any of them right now. Models still make stupid mistakes and people who do not read the code produced are bound for surprising discoveries in their codebase. Not trying to downplay the effectiveness of LLM-assisted coding and all that, but it's hard to imagine how much stupid shit i would've committed if I was not reading their output.
Serious question: Are you do a combative code analysis with a range of different models?
I used to find real dumb bugs in code reviews. With 2-3 different models doing a combative code review.. most of that is already fixed when I get it.
1 reply →
If you can test well enough, both manually and automated, then it's okay if there's a bunch of stupid mistakes in the code, as long as the end result works right. Not unlike biology.
1 reply →
> I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
I was doing the same (along with GLM 5.3 Flash) for months, and then gave Claude Sonnet 5.5 at medium effort a try. The difference was like night and day. Where both DeepSeek 4.1 Flash and GLM 5.3 Flash kept making mistakes and would have to do a bunch of iterations to find and fix them, Claude Sonnet 5.5 just did it right on the first try.
In my experience, while DeepSeek 4.1 Flash and GLM 5.3 Flash are a lot cheaper per token, they're just as or maybe even more expensive when you factor in all the extra tokens that are need to troubleshoot and fix their mistakes -- and even more expensive if I factor in my own time wasted doing the same, not to mention the frustration of having to deal with that.
DeepSeek 4.1 Flash also hallucinates a lot.[1]
I'm hoping all this improves with time, and the gap between the competent frontier models and open weight models narrows.
[1] - https://www.youtube.com/watch?v=P4dTq4X8bqk
I don't think it's a fair comparison. Maybe against deepseek pro would be more similar in terms of model size
Same for me. The cheap Claude subscription is just so much cheaper. Never had to buy more deepseek tokens for months (August), because Sonnet 5.5 is good enough
That’s the Model Welfare speaking, not your bias.
OpenAI's entire business is trivially profitable when you attach an ad network to ChatGPT.
Quick, someone point out the one billion user system with intent + personal data rich that doesn't result in a $100+ billion high margin ad network.
It is that simple and absolutely nothing can stop the gigantic ad networks that will be built up around personal agents.
Only if CPM can be high enough to counter balance the inference costs.
Current CPM is between $30 and $60 which is x20 to x30 more expensive than alternatives.
Now we have to wait and see if performance is also at least 30x of previous ad targeting.
What is that CPM? My CPM in ChatGPT is lower than in Google. CPM is a bad metric, how much does 1 conversion cost? You can charge me $100 per ad all day, as long as every showing results in a $1000 sale.
There's a trillion plus dollars (per year) of ad business up for grabs in the LLM era.
Facebook is toast, sooner than later, as a business. Whatever gradually replaces/upgrades that quasi dead friends network, will be premised on AI (Zuck apparently thinks it's at least in part personal agents). In the present Meta still has a $228 billion ad business that is hyper profitable.
Google + Meta = ~$650 billion hyper margin ad business. OpenAI, aka LLM as a service for the masses, is going to want to take as much of that as they can. OpenAI's active user base comes at an attention cost to the other mega services like FB or Google Search, and ultimately that attention is worth ad dollars. The question is only whether OpenAI can take it, not whether it's there at all or high value. And of course there's no guarantee that it'll be ChatGPT upon which the next great ad network is built, it will be some AI-centric business however, and likely heavily involving LLMs.
Google's operating income last quarter was $40 billion. If OpenAI fails to take the mass market + ad tandem, then Google will more likely retain their position. The solution to the cost is already well understood by eg Google: the masses don't need tip of the frontier AI capabilities, and relying on Nvidia's extremely over-priced GPUs is a dumb business model. The biggest threat to OpenAI right now is Nvidia's margins, not China. Google's focus has been on high quality cheaper AI systems for a reason: they're looking to serve billions of people that have no use for what the bleeding edge offers vs a fast + good mid-tier. Google can do that in a highly profitable way.
The thing that stops it is, ironically, personal agents. Get one to interact with the ad-driven one if you need to and filter ads out.
I suppose the very reason why we haven't seen ads yet is that they're trying to figure out how to prevent exactly this.
What? Ad networks REQUIRE a moat. Google Meta Amazon have historically defensible moats. ChatGPT? Zero moat. Good luck with that ad business!
A billion users and the extreme cost of infrastructure to compete are a gigantic moat.
You can't get the first and you can't get the money for the second. Nobody is going to give you $100 billion to underwrite the cost of competing with GPT at a one billion user scale (or even a 100 million user scale). The easy money is already drying up, the field is set, and you will not be able to get anybody to give you a huge check to attempt to chase OpenAI, Google, Anthropic, China. That's already over. No VCs will fund you on this particular business pursuit.
You couldn't meaningfully compete with OpenAI and Anthropic if you wanted to. There's absolutely nothing you can do about it.
There's a trillion dollar valuation waiting for you. What's up?
Even nation-state pockets Microsoft can't compete. If there were no moat, Microsoft would have their own GPT killer, and GPT would be dead. Microsoft has $160 billion per year in op income. They can freely afford to spend more than the entire EU (is willing to) on AI matters if it makes sense. They can't compete no matter how much they spend and they know it.
Google has nation-state money. They have chips. They have software developers by the zillions. They have a frontier or very near frontier model in Gemini / Argon. They invented a lot of the category. They can't kill GPT. I repeat, no matter what Google has thrown at the matter, they have failed to kill or stop GPT.
Google can't beat them. Microsoft can't beat them. China's models haven't stopped them (DeepSeek was supposed to kill them years ago; Kimi was supposed to kill them).
That's one helluva non-moat. They must just be really lucky.
You might as well claim Coca Cola has no moat because it's just sugar water. Brand at scale is a large moat. People use what they know unless you severely disrupt them. They're being taught to use GPT, they know GPT, and so long as OpenAI doesn't screw it all up, people will just keep using it. To say nothing of the fact that GPT still has one of the elite frontier models, so you also still have to compete with them on quality.
[dead]