Comment by mrandish

> a driving factor will likely be how good we can make models that are so small they use almost no compute.

That will certainly help but it doesn't move the fundamental limit because resource efficiency is a cost driver not a demand driver - and my argument is against the thesis that lying beyond professional devs and knowledge workers, there's an untapped trillion dollar industry serving LLMs to mass global consumers.

Using Simon's cost estimates, I agree that halving the current $1,000 - $1,200/mo MSRP to profitably serve frontier inference to professional developers and knowledge workers (PD&K) will help Vendor A steal share from Vendor B or C. It will also increase LLM sales penetration into the segments of the global PD&K TAM which can't afford ~$1K/mo for every seat. A fair chunk of the PD&K workers in many SMEs aren't included in today's ~$1K/mo per seat license pool, especially in 2nd and 3rd world geos. When the price falls to $500 and $250 most will but that's still just saturating the existing PD&K TAM - not pushing into mass consumers.

While the PD&K TAM is big, justifying Trillion+ dollar capex spend requires believing the TAM is much more than PD&K and eventually grows into converting a couple billion non-PD&K consumers into ~$20/mo subscribers. I don't buy it for two reasons:

1) The Comps: There are vanishingly few examples of long-term, mass consumer adoption of a discretionary technology at that scale. Mobile phones at ~$15 to $30/mo are the obvious one but LLMs are nowhere near being that valuable to the average plumber in Des Moines, baker in Jakarta or retired nurse in Hamburg. Pondering it, I just imagined forcing any of those people to choose between their mobile phone and an LLM chatbot. Sure, some who are flush with cash might choose both but for most consumers in the world ~$20/mo is big enough they'd have to pick one and ~zero percent would choose the LLM over their phone. After mobile phones, the second comp for discretionary tech spend I thought about was XBox and Playstation monthly gaming subscriptions but combined they have less than 90M paying subscribers and the ARR is just under $10/mo. As an industry, "Big LLM" is spending well over a trillion dollars every five years. XBox and PS ARR doesn't even cover paying the interest on that capital, much less the 3 to 5x returns hedge fund investors are betting on.

2) The Alternative: It's useful to doubt my own intuitions and one counter to my skepticism is to assume "But LLMs aren't finished yet, they're going to get much better." How much better could an LLM which can be profitable at ~$20/mo get than Claude Mythos in the next five years? Instead of debating future unknowables with myself, I've found it's better to just imagine the most perfect future product I can that's still realistically plausible. So, let's imagine we're willing to spend a million dollars a month to very unprofitably deploy a prototype to test the consumer demand for "Tomorrow's Awesomest $20/mo LLM" today. So we gather a few hundred super smart, broadly knowledgeable intellectuals together at one top-tier university research library, where they'll have access to every commercial database and unlimited Claude Mythos 2.0 and ChatGPT 6.0. Since our experimental budget is $1M/mo we can afford to add in several Nobel prize and Fields Medal winners too. They'll work together manually reviewing and improving not only every LLM answer but also our test user's prompts - and of course our test chatbot will have human-level real-time speech recognition and vision (via Zoom and screen-sharing with actual genius-level humans), making this truly a test of the "smartest, most accurate, best consumer chatbot" we can imagine.

Now, let's run the test by having one thousand mass consumers try it out and see how many Des Moines plumbers, Jakarta bakers and Hamburg retired nurses we can convert to a 1 year @ $20/mo subscription for our $1M/mo ultimate chatbot simulation. Playing this thought experiment out in a bunch of ways, I find some percentage of outliers, iconoclasts and closet intellectuals would go for it but... the vast majority just don't find it enough better than "free" chatbot alternatives AT&T includes with their phone subscription or Samsung bundles with Galaxy phones - despite only being ChatGPT 5.4-level. It turns out, most plumbers, bakers and ex-nurses don't have a compelling "job to be done" in their daily lives that even an MoE panel of actual Nobel and Field's medalists with ivy league professors can make enough more valuable than an inferior but free-to-me chatbot, in the judgement of our Des Moines plumber. While the world's smartest chatbot is nice, when it comes time to pay, he prefers having one additional premium football match on TV and a six pack of cold beers every month.

> What are the non-tech people in your life using AI for?

Mostly asking random questions they used to search Google for.

> next to Starbucks and avocado toast, is discretionary.

Sure, but your description implies highly affluent, urban professionals in western nations. I was talking about getting several billion global mass-market consumers to all keep paying ~$20/mo. Mass consumer adoption of mobile phones worldwide is currently >5.8 billion or >70% of humans alive. Only ~50M people are paying $20/mo for an LLM and I suspect many of them are not pure consumers but actually knowledge workers that AI vendors are losing money on and will eventually force into higher tier plans just like the $200/mo developers they're currently losing money on. These heavily subsidized loss-leader offers are all going away post-IPO.

Personally, I know maybe a dozen people who pay $20/mo for an LLM but only two of them are really 'pure consumers' who don't use it for knowledge work. Both of them are multi-millionaires and neither has had a job in ten years. One is retired like me and the other is so wealthy she has a Netjets credit card and has new cars delivered like some people order shoes. Everyone else I know paying $20/mo is a professional who uses the LLM for a lot of office or knowledge work and writes it off as a business expense - examples include a couple of attorneys who are senior partners in a law office they own, a solo architect, and a dentist who owns his own practice.

At $20/mo, AI vendors are probably losing money on most of my professional friends because they use it pretty heavily all day. They're only making money on the two multi-millionaires who both use it so infrequently they could easily be using free chatbots instead but are so rich they could lose $10,000 in their couch cushions and not notice. While they are profitable at $20/mo, they aren't exactly "typical consumers" that there are billion more of. I expect AI vendors will find ways to force my lawyer, architect and dentist friends to switch to higher priced plans soon because they're really knowledge workers abusing a consumer tier plan into unprofitability.