Comment by jredwards
6 hours ago
It's tough for me to square the two things happening simultaneously in AI right now:
1. LLM Model providers are starting to charge real costs to users, revealing that AI usage is much more expensive than the subsidized rates we've been seeing for years.
2. Google is now using an LLM to answer every single google search that happens, for which Google bears the entire cost.
In my experience, LLM usage follows an exponential distribution i.e. most people using LLMs are not using many resources, but a very small few are using a massive amount of resources. Most LLM usage is trivial and tiny, most of what I use LLMs for could be done by a local model with a decent web retrieval tool. What some people I know use LLMs for requires massive volumes of tokens. Its that small power user base which I would bet they are targetting.