Comment by root_axis
2 hours ago
I think the points you raise are reasonable signals to consider, but I don't think they show the author being "consistently wrong". The overall thesis still remains plausible even though we have seen LLMs continue to improve.
> - hallucinations are dramatically less of a problem
Sure, but it remains a big enough problem that human intervention and review is still necessary for any serious work across all use cases and industries.
> - several mass market use cases have emerged, most notably coding
Coding seems to be the only one, but there are still a lot of open questions about how the market can sustain the costs, and that's without considering the market dynamics that could emerge once costs are lowered enough that open source models start to become an attractive option.
> - rate of progress has increased
Debatable.
> Sure, but it remains a big enough problem that human intervention and review is still necessary for any serious work across all use cases and industries.
Another important consideration: Hallucinations getting less common/severe but not (as-good-as) solved makes them worse.
LLMs used to very obviously get things wrong. And people wouldn't trust them. Now they're good enough that people blindly trust them.
Now people just directly PR AI output with little to no manual review. We even have clowns calling for the complete abolition of directly human-authored code.
Whatever gains were had in better AI code output over the past two years I lose in having to review much more thoroughly.