Comment by antupis
2 days ago
Kinda interesting to see where the moat is in AI space. Good base models can always distilled when you have access to API. System prompts can get leaked, and UI tricks can be copied. In the end, the moat might be in the hardware and vertical integration.
> the moat might be in the hardware and vertical integration.
The moat is the products that can be built. The moat is always the product - because a differentiated product can't be a commodity. And an LLM is not a product.
Google and MSFT and Meta have already "won" because they have profitable products they can build LLMs onto. Every other company seems to be burning cash to build a product, and only ChatGPT is getting the brand recognition to realistically compete.
Building an LLM is like building a database. Sure a good one unlocks new uses, but consumers aren't buying something for the database. Meanwhile enterprise customers will shop around and drive the price of a commodity down while open source alternatives grow from in-house uses to destroy moats.
Even hardware isn't a true moat. Only Google has strong vertical integration with their TPUs, and that gives them a lead. BUT Microsoft, AWS, Meta and a whole bunch of startups are building out custom silicon which will surely put pressure on them and Nvidia to keep innovating and earning that price edge.
See I kind of buy the database argument but also kind of don't. A database needs an operator whereas a LLM doesn't. You're basically melting the product into a piece of goo and the UI can be approached using natural language.
For products that still need a UI you could claim that LLM operators take over, so that's still a tax you pay to the incumbents as you interact with a product. It's sort of like we take the money which was paid to SQL operators and engineers and instead pay it to the hyperscalers.
LLMs absolutely need an operator - who runs the servers and GPUs that hosts the models? Who writes the system prompts? Who fine tunes and trains the models? This can be a big cloud api like AWS, but it can also be a custom-in-house service for a company.
Users of LLMs don’t quite have an equivalent employee to a DBA, but neither do most customers of AWS DynamoDB or RDS or whatever.
Many use cases of LLMs won’t be chat bots like ChatGPT. They’re be tools for automated summarizations, classifications, etc. They’ll be automated assistance and basic tool calling, etc. They’ll perform OCR and documentation analysis. Automated translations etc.
Oracle is doing great just selling databases. Having your data is a moat.
How many times have we been down this path? Tcp/IP, dos/windows, Linux, virtualization, and on and on. Open platforms always seem to find a way to usurp everyone else. In the end, it's better to be a service provider.
Open source finds a way.
Good enough + open (and free) is a very appealing proposition.
> Good base models can always distilled when you have access to API.
What does that mean?
You can use the outputs of a closed source model (or deepseek -> llama. see llama 70b deepseek distilled) to create a synthetic training data set which lets you fine tune (distill) most of the benefits of the "smarter" model in to a "dumber" model. This is why openAi does not show the actual full chain of thought but a summarized version. To stop exfiltration of their IP which has proven immensely difficult.*
*disclaimer; i am an expert of nothing
Why do we need a moat?
_We_ don't. Investors do. Because without being able to gatekeep the rest of the world, there is little money in LLMs.
Indeed.
I guess investors should stop pouring money into LLMs, then. Just like how they don't pour money into pure mathematics.
So a company can make enough money to fund the next breakthrough/training run
there is no open source alternative to GPU farm, that's the moat
that's why they can open source their model and be fine because running this shit is actually hard, let alone maintaining SLA for millions of users??
How long until laptops are able to run high end models? What's the use case that requires a server farm for end user's?
maybe next 5 - 10 years??? but even then the frontier would be push further and people would get used to lets say 10 trillion model cloud host and using 600B model would feel stupid
>Kinda interesting to see where the moat is in AI space.
Where we're going, we don't need moats.
ecosystem