Comment by phaser
6 hours ago
What intrigues me the most about AI progress, is not AGI or the model du jour by $AI_UNICORN, but rather what can be run locally. I remember having an amusing, but rather useless model in a beefy gaming PC that I had 6 years ago; and now, something that’s a hundred times better on my M5 laptop.
Should the market react to the memory shortage, the progress of the Apple silicon continue at the same pace, and what we’ll be able to run locally in 6 years will be very exciting. or frightening.
Also I don’t know what this means for the valuation of the AI companies. I remember asking about this very idea to one of their employees at an event and instead of answering he bailed out to grab a cocktail.
Things you are not supposed to talk about:
- There is no "moat" (lasting, easy-to-defend technological edge) in AI model businesses. There are just short-term advantages.
- An AI business is a capital-intensive business, just like old factories. Data centers are expensive, models are energy-hungry, and the hardware inside must be replaced every 3–4 years.
- Smaller, specialized models eat margins from below. Transcription, voice, or image detection do not need large models.
There is no reason to expect high margins like you can in traditional software business. Benefits of AI go mostly to consumers.
edit: There is potential for economies of scale. Few megacorps can strive for cost advantage when they achieve scale (Microsoft, Google, Amazon and Meta)
Everybody talks about this stuff all the time
All true.
It does seem like the structural characteristics we’ve observed so far suggest there is a kind of flywheel from short-term to long-term advantage due to the capital requirements at various levels.
If you’re Nvidia, making the best GPUs today, the expanding wavefront of demand is consuming them with volume and margins to give you a huge edge in building out the best next generation of GPUs. Similar to how the mobile wave gave TSMC sustained advantage for about a decade now.
I’m guessing this is also what we’re seeing as Anthropic and OpenAI swap spots in the token-vendor market.
I can see the fly wheel in action for Nvidia[1], but in terms of model building - I think the companies that have the advantage here are not Anthropic or OpenAI, but rather companies with substantial revenues from other sources - Google is the obvious player here - reported to be planning on spending 185 billion this year without having a raise a dime from the markets, but there are plenty of other companies - like Meta or Alibaba who can easily fund the longer game from existing revenues.
What you can run locally in consumer hardware is progressing pretty well.
If you get a not-quite-the-best gaming GPU like a 5080, you can run local models that are better than the state of the art from early 2025. Depending on what you want to do, you might have to switch models. The one size fits all huge models are still a data center thing.
--what this means for the valuation of the AI companies
Probably nothing. Most users have no idea what an LLM is or how it runs. Anecdotally speaking, I see many LLM users default to whatever their day job provides to them. And even slightly more sophisticated users seem ok with paying for their openai or anthropic subscriptions.
Maybe we will see a small but dedicated group of open weight model users who prefer local llm, but everybody else will just consume from the big providers? The scenario might look something like OS choices today - a small, committed group of Linux users vs the vast majority of other users running Windows, MacOS, or Chrome?
Its a convenience thing. You can run a whole lot of stuff locally from wikipedia to social media/email/video servers whatever. Most people with a full time job and 2 kids dont do it cause who has time and energy to patch and maintain the ever growing complexity of this stuff. These systems will keep growing complex. That also means more bugs. Age old tradeoff between freedom and convenience.
You can run mediawiki at home but you won't have wikipedia. You can run a video server but you won't have all the movies that Netfix has. A local model is actually the real thing.
you can have the whole wiki loaded with full search available locally. check out kiwix.
[dead]
This has always been true of software, particularly games. You can get a 5-6 year old game for a fraction of the price, and run it on modest hardware. But the industry wont sit on its hands for 5 years, there will be newer software that requires better hardware.
Technology doesn't always work like that.
A new game is a totally new world with everything created from scratch. A creation. A model, on the other hand, is a reinterpretation machine for hundreds of years of human creations, but not a creation in itself, more like a discovery.
You would think that by now we would have a much better Bitcoin that's taking over the payment networks of the world but what we actually got is a shitload of shitcoin.
Training AI models to drive valuation reminds me of high frequency trading