Comment by apparent
7 months ago
This is the result of the winner-take-all (most) economy. If the very best LLM is 1.5x as good as good as the next-best, then pretty much everyone in the world will want to use the best one. That means billions of dollars of profit hang in the balance, so companies want to make sure they get the very best people (even if they have to pay hundreds of millions to get it).
It's the same reason that sports stars, musicians, and other entertainers that operate on a global scale make so much more money now than they did 100 years ago. They are serving a market that is thousands of times larger than their predecessors did, and the pay is commensurately larger.
I don't think AI will be a winner-take-all scenario. If that is to happen, I think the following assumptions must hold:
1) The winner immediately becomes a monopoly
2) All investments are directed from competitors, to the winner
3) Research on AGI/ASI ceases
I don't see how any of these would be viable. Right now there's an incremental model arms race, with no companies holding a secret sauce so powerful that they're miles above the rest.
I think it will continue like it does today. Some company will break through with some sort of AGI model, and the competitors will follow. Then open source models will be released. Same with ASI.
The things that will be important and guarded are: data and compute.
Yeah, this is why I said "(most)". But regardless, I think it's pretty uncontroversial that not all companies currently pursuing AI will ultimately succeed. Some will give up because they aren't in the top few contenders, who will be the only ones that survive in the long run.
So maybe the issue is more about staying in the top N, and being willing to pay tons to make sure that happens.
>I think it's pretty uncontroversial that not all companies currently pursuing AI will ultimately succeed.
That's probably true, but at the moment the only thing that creates something resembling a moat is the fact that progress is rapid (i.e. the top players are ~6-12 months ahead of the already commoditized options, but the gap in capabilities is quite large): if progress plateaus at all, the barrier to be competitive with the top dogs is going to drop a lot, and anyone trying to extra value out of their position is going to attract a ton of competition even from new players.
I agree with this comment.
Maybe it's just me but I haven't been model-hopping one bit. For my daily chatbot usage, I just don't feel inclined to model-hop so much to squeeze out some tiny improvement. All the models are way beyond "good enough" at this point, so I just continue using ChatGPT and switching back and forth from o3 and 4o. I would love to hear if others are different.
Maybe others are doing some hyper-advanced stuff where the edging out makes a difference, but I just don't buy it.
A good example is search engines. Google is a pseudo-monopoly because google search gives obviously better results than bing or duckduckgo. In my experience this just isn't the case for LLM's. Its more nuanced than better or worse. LLM's are more like car models where everyone makes a personal choice on which they like the best.
I agree with you, and think we are in the heady days where moat building hasn't quite begun. Regarding 1) and 3), most models have API access to facilitate quick switching and agentic AI middleware reaps the benefits of new models being better at some specific use-case than a competitor. In the not-so-distant future, I can see the walls coming up, with some version of white-listed user-agent access only. At the moment, model improvement hype and priority access are the product, but at some point capability and general access will be the product.
We are already seeing diminishing returns from compute and training costs going up, but as more and more AI is used in the wild and pollutes training data, having validated data becomes the moat.
> Right now there's an incremental model arms race
Yes, but just like in an actual arms race, we don't know if this can evolve in a winner takes all scenario very quickly and literally.
> just like in an actual arms race
In an actual arms race you use your arms to actually physically wipe out your enemy.
It's not just like an arms race.
3 replies →
The problem is models are decaying at incredible speed and being the leader today has limited guarantee you’ll be it tomorrow.
OpenAI has a limited protective moat because ChatGPT is synonymous with generative AI at the moment, but that isn’t any more baked in than MySpace (certainly not in the league of Twitter or Facebook).
> I don't think AI will be a winner-take-all scenario.
AI? Do you mean LLMs, GPTs, both, or other?
Why won't AI follow the technology life cycle?
It'll always be stuck in the R&D phase, never reach maturity?
It's on a different life cycle?
Once AI matures, something prevents consolidation? (eg every nation protects its champions)
The actual OpenRouter data says otherwise.[1] Right now, Google leads with only 28.4% marketshare. Anthropic (24.7%), Deepseek (15.4%), and Qwen (10.8%) are the runners-up.
If this were winner-take-all market with low switching costs, we'd be seeing instant majority market domination whenever a new SOTA model comes out every few weeks. But this isn't happening in practice, even though it's much easier to switch models on OpenRouter than many other inference providers.
I get the perception of "winner-take-all" is why the salaries are shooting up, but it's at-odds with the reality.
[1] https://openrouter.ai/rankings
Openrouter data is skewed toward 1) startups, 2) cost sensitive workloads, and generally not useful as a gauge of enterprise adoption
But those are just the sort of cases that are more likely to switch to the latest, greatest and cheapest when they come around. The fact that even these haven't become winner takes all is a strong signal that these models have sticking power.
3 replies →
Is there good public data on enterprise adoption?
I and most normal people can’t even tell the difference between models. This is less like an arms race and more like an ugly baby contest.
Actually that is decent data and reflective of current SOTA in terms of cost performance tradeoff
I don't think so. It's just a bubble. There's no AI, we have fancy chatbots. If someone were to achieve AGI, maybe they win, but it's unlikely to exist. Or if it does, we can't define it.
What would you say if the IMO Gold Medal models from DeepMind and OpenAI turn out to be generalizable to other domains including those with hard-to-verify reward signals?
Hint: Researchers from both companies said publicly they employ generalized reasoning techniques in these IMO models.
Nice. That's really great progress. Maybe a model will be able to infer what consciousness is.
> There's no AI, we have fancy chatbots.
"Fancy chatbots" is a classic AI use case. ELIZA is a well-known example of early AI software.
I'm not familiar with ELIZA. Technical terms mean things. AI has no technical meaning. It's difficult to distinguish what people mean when they say AI, but as technical people I assume we know AGI doesn't exist yet and we don't know if it ever will.
If the next best model is 0.5x as expensive, then many might opt for that in use cases where the results are already good enough.
At work we are optimising cost by switching in different models for different agents based on use case, and where testing has demonstrated a particular model's output is sufficient.
I don’t see a winner takes all moat forming. If anything, the model providers are almost hot-swappable. And it seems the lifespan for being the best SOTA model is now measured in weeks.
It's true they are currently quite interchangeable these days. But the point is that if one can pull far enough ahead, it will get a much bigger share of the market.
> winner-take-all (most)
> If the very best LLM is 1.5x as good as good as the next-best, then pretty much everyone in the world will want to use the best one
Is it? Gemini is arguably better than OAI in most cases but I'm not sure it's as popular among general public
I don't think there's a consensus on this. I have found Gemini to be so-so, and the UX is super annoying when you run out of your pro usage. IME, there's no way to have continuity to a lower-tier model, which makes is a huge hassle. I basically never use it anymore.
The hack I found to “get around” the secret limits that pop out of nowhere is to you export the chat, then import it with another account. So you don’t need to pay 200, just 40 (and only when you actually need that much).
In other words, what matters is not just which one is "best"?
1 reply →
It's multivariate; better for what? None of them are best across the board.
I think what we're seeing here is superstar economics, where the market believes the top players are disproportionately more valuable than average. Typically this is bad, because it leads to low median compensation but in this rare case it is working out.
> If the very best LLM is 1.5x as good as good as the next-best, then pretty much everyone in the world will want to use the best one.
Well only if the price is the same. Otherwise people will value price over quality, or quality over price. Like they do for literally every other product they select...
Most likely won't be a winner-take-all, but something like it is right now, a never ending treadmill where the same 3 or 4 players release a SOTA model every year or so.
Who the heck is making profit?
[dead]
That is exactly how all of this has played out so far (beep, not at fucking all!)
You are not one random hyperparameter away from the SciFi singularity. You are making iterative improvements and throwing more compute at the problem, as are all your competitors, all of which are to some degree utterly exchangeable.
There is LLM 1.5x as good as the next best.
I tried multiple and they all fail and some point so I let another LLM take over.
As soon it’s not some boilerplate thing it becomes harder to get the correct result
You’re right.
It would be unfortunate if something like Grok takes the cake here.
Why?
Because it's MechaHitler?
2 replies →