Comment by geraneum
5 hours ago
Do you have any references for the models deployed in any of these organizations?
> I suspect it's just not really possible for academic research
But we were told everything changed in fall-winter of 2025 and that when we got “real capable” agents/models. Whatever happened to that?
I have my own experience to go off of, but this can also be inferred easily. There are many large corporations still working on "digital transformation" from 10+ years ago. There's no way such companies have fully onboarded and integrated frontier AI agents throughout their workforces with high adoption. The paper itself reports low adoption even after conditioning on access.
Everything did change after late 2025, but there are tiers of usefulness. Late 2025/early 2026 was the first time agents became clearly useful for personal projects. That's distinct from immediately having economically valuable end-to-end autonomous workflows in teams of median talent, which many would suggest is the threshold we've just recently crossed.
If you want to refute a study which presents numbers and statistics, you have to provide it in the form of numbers and statistics.
I don't, actually. I can simply point out that its data is out of date, and then the numbers themselves are called into question without other numbers to compare them to. And that's what I did: I shared two arguments in two comments that converge on the shared conclusion that the paper's data is not up to date (or, more precisely, cannot be up to date).
2 replies →