Comment by RevEng
14 hours ago
I usually agree with Simon but I think he is overlooking an important factor.
There is a lot of AI usage happening not because it shows benefits, but because the business has mandated its ubiquitous use. Companies having dashboards for token usage and rewarding people for using more tokens is a real thing. I just spoke with someone today who works at Microsoft and they are required to use AI for all of their work - they have to make a special request with justification if they decide not to use AI for even a single PR. This kind of demand isn't driven by value from either the company itself or from its workers; it is the kind of artificial demand you get from make-work projects to keep people employed during hard times.
We have to wait for the hype to settle down and people start making business decisions based on results before we can really value these AI products.
Correct the question is one of real vs fake demand curves.
I think token leaderboards are idiotic, however… many companies are requiring AI use, with by far the most likely reason being that they truly believe AI-assisted development gets (or will after the time needed to get employees acclimated) better returns.
That mere belief by companies is not enough on its own to substantiate durable market fit. Companies believe all kinds of silly things. We're looking for long-term trends, and the returns have to be real for AI development to be more than a dream companies will eventually wake up from.