Comment by hedora
5 days ago
What moat? There are multiple companies providing pareto-optimal frontier models, and it takes O(10) people to build one of these things.
The rest is capital intensive, and the price will approach the cost of production over time.
Thinking this is a profitable endeavor is equivalent to claiming coal plants have good margins because boilers are expensive.
I think we agree?
What moat? You answered yourself: "capital intensive"
But, history says the supercomputer of today will fit in your pocket in a few years.
They've bought up all the RAM and GPUs, which pushes the capital requirements upward for everyone else. But, they can't corner the market forever, there are too many competing interests. AMD and Intel keep making new GPUs and APUs. The memory makers can't just sell to only AI companies forever, if they do Chinese manufacturers will move in and eventually eat them from below (as has happened many times before).
They have a moat today, and it's just that it's really expensive to train and host frontier models, especially at commercial scale. It used to be there was also some secret sauce to making it fast and efficient. But, secret sauce is being published daily by all sorts of researchers, folks are figuring out how to do more with less and it often finds its way into llama.cpp or vLLM or SGLang within days or weeks.
> But, history says the supercomputer of today will fit in your pocket in a few years.
I don't think this will be true in the same time span anymore. Each miniaturization is costing more and more money.
Perhaps they'll come up with exotic fundamental improvements, but I don't think the rate of improvement of compute/watt will match the previous decades.
Yeah, that's probably true, but we're also seeing that there's still tons of inefficiencies in how LLMs are being run. Seems like every couple months there's some new technique to squeeze more performance out of less hardware. KV caching improvements, fast attention, speculative decoding, dynamic quantization, quantization aware training, etc.
That said, I recently replaced my five year old self-built PC (with a top-of-the-line desktop CPU, chipset, memory, and GPU of the time) with a new everything-the-best build, and while it's clear we're not keeping up with Moore's Law anymore, it's still 4-5 times faster for compute-intensive stuff, especially parallelizable tasks. We're still getting faster/cheaper. So, the time scale is maybe ten years rather than five.
1 reply →
Really the biggest concerns are not computers getting spectacularly faster, but 'intelligence' algorithms getting orders of magnitude better.
Drop the power requirements 1000 fold, and yea you will be able to make your own SOTA model on the cheap. The problem is the person that has a few exaflops of power will still leave you in the dust in the intelligence explosion that would happen after an event like this.
6 replies →
Single clock speed hasn't had much of an upgrade, but the architecture for doing exactly what they are doing? That will improve for at least 5-10 years. There are both huge power gains from Processing in Memory (PIM) chips (70-80% discount in energy), and improvements to engineering to make memory cheaper and cheaper.
3 replies →
That has never been true, unfortunately. The 2005 top500 was led by bluegene/L achieving 280 FP64 TFlop/s.
Apple is talking about 17.5 FP16 TFlop/s on the iphone 17 neural engine. So 20 years later we are still nowhere near, not even at reduced precision.
2 replies →
In five years I think you will be able to train a frontier modem for much less money than today and the power hungry hardware of today will be cheap second hand due to the power usage.
1 reply →
>but I don't think the rate of improvement of compute/watt will match the previous decades.
Unless we invest heavily in research and find new way to do chips. But I think there's not enough motivation and money to do that.
1 reply →
> I think we agree?
That is such a crazy way to start a response to someone trying to argue with you. I should try this. That's amazing. I know you didn't mean it as a trick, at least I'm pretty sure you meant it sincerely, but I'm just struck by the power of it to defuse and redirect the conversation. And this was a very low-grade example, but I could imagine this being useful in much more heated contexts.
I think in general stripping away the parts you agree with from the argument works great, because it strips away a whole lot of potential for ending up indirectly arguing over things that aren't in contention, and it often also defuses the rest when it turns out the core of the argument perhaps is much smaller than people are willing to get invested in.
4 replies →
OTOH I have often witnessed people agreeing without realizing it. I‘ve been able to defuse a bunch of arguments by pointing that out.
In fairness I completely agree with 99% of their comment.
I was nitpicking the use of the word “moat”. For it to be a moat, it’d need to be more expensive to traverse than to build.
Instead, the big AI firms are trying to create a monopoly on capital in an area where real costs are dropping 90% year over year.
Usually people are taught these techniques at the management courses. If you're at a BigCorp where they push managers through such courses - you can hear a lot of that stuff in your manager's speech if you pay attention to it.
Yeah, more valuable than the comments I came to read (even if those are interesting too!)
[dead]
The other half of the moat is the data they stole from everyone else, some of it illegally. So, be sure they will do everything in their power to stop others from getting that data freely.
Yeah, I think a lot of the "slow down" rumblings we're hearing from OpenAI and Anthropic are really overtures toward regulatory capture; basically, "now that we're in the lead, we need to lock this shit down so nobody else can catch up."
4 replies →
They’ve bought up all the RAM and GPUs…
Is there an endgame where even this is considered overly complex? Instead of starving the competition by buying up all the compute, why not just buy up… all the money!? Hoover up as much investment capital as possible so that your competitors can’t get funding.
I assume this is an honest question, in which case the answer is funding is not really finite.
1 reply →
or just "buy" your competition like big tech did
every major tech company literally have deal,ownership,alliance etc
they literally not gonna gobble up entirely to trigger anti-trust case
They did get a bunch of investment grants from Trump, so your tax money (and power bills) are subsidizing them. They also arranged for ETFs to eliminate consumer protection rules to force everyone’s retirement to buy SpaceX/Anthropic/OpenAI shortly after IPO. That totals $3T in valuation (unless it goes up in first week trading), so your retirement is basically going to be weighing “AI bubble” similar to “MAGA”, and then everything else is rounding error. (The rule changes waive profitability requirements, and shorten the cooldown from IPO to indexing from a year to weeks).
I guess that’s one way to try to make capital finite.
>But, history says the supercomputer of today will fit in your pocket in a few years.
That was Moore's law saying that. And it seems Moore's law slowed down quite a bit for now.
Yes, but surely AI are going to save us from the bloated stack of modern software.
"But, history says the supercomputer of today will fit in your pocket in a few years."
hmm nooo ??, physic says otherwise
> it takes O(10) people to build one of these things
To build a working prototype, sure. To operate at production scale, definitely not. The same rule would apply to WhatsApp and many other world-scale products. Turns out that, the moment you need to monetize these machines, your O(10) stops working.
O(10) people?
So, a constant number of people.
(less facetiously, I think they mean "5 to 50")
Other models arent even close except for gpt 5.5. You're dead wrong on that. You read too many benchmarks and/or chinese propaganda. There hasn't been a serious contender in agentic SWE besides OAI and anthropic for a long time, and no chinese model has even reached opus 4.5 performance yet. The moat isnt insurmountable but it is very solid for at least a 12 month lead time. Which is such an insane amount of time in this landscape and industry. The moat is stretching, not shrinking, on agentic SWE. And that is literally the only moat that matters for RSI.
DeepSeek 4 Pro is performing agentic SWE tasks for me quite well. It can't do everything Opus can do, but if OpenAI and Anthropic disappeared tomorrow, I'd figure out ways to make it work with harness improvements and other optimizations.
Anthropic can stretch the moat all they want, but in the department of trust, they put a final nail in their coffin today. Anthropic is pure evil at this point.
'evil' lol. Every single corporation you deal with is evil then. it's greed. and almost every large model provider is guilty of it. China is all open source right now. cool! gee i wonder what would happen if they ever actually achieved SOTA? They would clamp down on that so fast Dadio's dradel would spin
5 replies →
I use gpt 5.5 at work (because they pay for it) and DeepSeek at home (because I pay for it) and while I do agree one is better than the other, I think you’re really overstating how far apart they are. Just my take.
What's 12 months lead time worth? Not much from what I can tell. Contrary to what these AI companies might tell you, if an AI model can't do it, a human can still do the work.
Honest question, is it possible that since might be using the latest/best model to analyze and improve the existing ones, the moat will expand exponentially, making the models better and more efficient at each iteration until there is no point in competing?
All models from the past two years are close in the general case.
This is just another incremental improvement, rushed out to boost the ipo, AI has the capacity to aid an engineer but this minor bump in performance will have essentially zero impact on the productivity of an engineer working on real world solutions when compared with any other major model.
We are trending towards asymtotic and it can't happen fast enough, that's when the true cost of this will become evident.
Most of HN is stuck in this fantasyland where they insist their local LLM setup is comparable to Opus 4.8 or GPT 5.5. It's like a collective delusion, I've never seen anything like it.
You can get really good results with Chinese models. You're putting Opus and GPT on too high of a pedestal.
3 replies →
Some of the new and open models are very capable now, The truth is, the value of the model is in the mind of the user - the big names are impressive to those who know little and are dazed by little, but they are bound to end up wrong regardless of how good the model is.
4 replies →