Comment by 0xbadcafebee

That's a short-term view of a subset of users.

Think about mobile users. If you compare the raw cost for MVNOs vs MNOs, they have basically the same cost. There is no special moat around text messages, voice calls, data. By law, MNOs have to give MVNOs access to resell their network access; it's literally the same network. Yet MVNOs are 2x-6x cheaper than MNOs. How is that possible? Trickery. MNOs hide fees, bundle "features", lock people into exorbitant prices, and increase costs over time. Nobody today sees the price per minute of calls, text, data. If you look really, really closely, you can compare data caps, international vs domestic. But almost nobody does.

Now back to LLMs: Normal users do not pay API cost, they pay subscription cost. They're going to be the vast majority of users. No normal user is going to look at API provider costs. They're going to pick one of a handful of subscription providers. Those subscription providers will hide the real costs behind the scenes, making deals with inference providers, who will in turn hide those prices for competitive advantage. They will raise prices, and it'll be buried in the subscription fees. This has been happening for a decade with mobile subscriptions, and it's already happening now with LLM subscriptions. (There is no price-list for current LLM subscriptions; it's a total mess that's constantly changing)

Enterprise and government are large-scale users, and they'll pay exactly the same way they pay for compute right now (LLM cost is literally compute cost). Some will self-host, some will pay API pricing, some will rent, some will use "team" or "business" subscriptions. This is also all happening right now. AWS is making bank from Bedrock purely because people in their ecosystem would rather spend money through a known vendor with marketplace credits, and that's just one of a hundred different examples.

To put it in simpler terms: AI is just servers. Normal people do not pay for servers, they pay a service provider for services. Businesses pay for servers, but in many different ways, and definitely not the cheapest options.