Comment by raincole
2 days ago
Any research on the impact of AI would have been lagging indicators. It's not the researchers' fault[0], but the nature of a field moving at neck breaking speed. Remember there was a paper saying programmers were 20% slower with AI?
[0]: well...
That early 2025 METR study was particularly interesting because participants self-evaluated themselves as 20% faster, but the measurements showed they were actually 19% slower.
All the reports of productivity since then are self-reported, or using questionable measures such as SLOC and PRs, so it’s reasonable to say that productivity improvements are still unknown.
Unfortunately, METR hasn’t been able to replicate the study because they couldn’t find enough willing participants.
> Remember there was a paper saying programmers were 20% slower with AI?
Key thing was not that they were X% slower, but that they were slower while being convinced they are faster. Of course, any analogies with the current hype cycle are completely unfounded.
Well… what