Comment by mrandish
1 day ago
Glad to see more data and discussion of this. Lately the gap between ambient belief that "LLMs can now replace ~90% of all 'knowledge work'" and actual reality seems like it's growing.
I've found discussions on HN tend to quickly get pulled toward polar extremes between those who are justifiably impressed at the increasing effectiveness of LLMs and those who highlight the brittle jaggedness and counter-intuitive gaps of LLM 'intelligence' in daily practice. One side keeps citing more examples of 'uniquely human traits' LLMs don't seem close to replicating while the other complains of eternally moving goalposts.
Today I think both extremes are partly right but also partly wrong, and end up largely talking past each other because so much remains ill-defined, uncertain or currently unknowable. IMHO, the most accurate position is a carefully nuanced middle-ground built on well-defined scopes, terms, time frames and probabilities with frustratingly wide error bars. Partisans at either extreme seem uninterested in more than drive-by hot takes.
It can be simultaneously true that "Someday AI will be able to functionally replace all human knowledge work" and "That day does not appear to be soon, and even when it arrives, there will still be uniquely human traits which are valuable and that LLMs can't satisfactorily replicate."
All this is saying is that there's a lot of stuff benchamrks aren't covering. It's not saying models can't do those things well or don't get used to do those things.
-> the most accurate position is a carefully nuanced middle-ground
Nuanced position will not be middle ground. Single-dimensional thinking is exactly the problem that causes the argument. There are multiple reasoning modalities, each suitable to a certain type of human/agent cooperation, and there are different tasks where different modalities are applicable at different stages. There is no way to reduce this to a single axis.