Comment by JacoboJacobi
18 hours ago
I'd like to see a neutral productivity measure? Whether you tell me it went way up or way down I tend to be suspicious of productivity measures being neutral to perception changes that effect expectation, non paradoxical, etc.
It makes a lot of intuitive sense: people feel more productive because they're twiddling switches but they're spending so much time on tooling it doesn't actually increase output (this is more or less what the MIT study found: 20% perception of productivity, 20% lower actual output).
Sure but increased output would mean code. I don't think generating a lot of code is itself developer productivity. Some people could be using it to stop themselves from creating bad code which is developer productivity. While I find it a bit unlikely people are using it in this way (in terms of the average) I would most certainly have made this argument if code quantity was up from LLMs so I can't claim to know a quantitative measure.
My hypothesis for why our developers have reduced productivity is that LLM assisted coding has made reviews much more difficult. The words that are written are subtly more complex for a human to understand compared to what our engineers would have previously written themselves. Sort of an uncanny valley effect.
Couple that with engineers across the board mentioning that they feel like they're losing proficiency in an understanding of the codebase and where things are.
The model that does make sense to me is (and the only actual success stories I've seen) is people saying "it let me quickly produce a piece of software that otherwise wouldn't have been worth the time to create". That is definitely an increase in productivity, but "software people aren't actually willing to pay for can now be made much more cheaply" is a much different claim than the marketing is making (which I read to be TFA's point).
1 reply →