Comment by JacoboJacobi
14 hours ago
Sure but increased output would mean code. I don't think generating a lot of code is itself developer productivity. Some people could be using it to stop themselves from creating bad code which is developer productivity. While I find it a bit unlikely people are using it in this way (in terms of the average) I would most certainly have made this argument if code quantity was up from LLMs so I can't claim to know a quantitative measure.
My hypothesis for why our developers have reduced productivity is that LLM assisted coding has made reviews much more difficult. The words that are written are subtly more complex for a human to understand compared to what our engineers would have previously written themselves. Sort of an uncanny valley effect.
Couple that with engineers across the board mentioning that they feel like they're losing proficiency in an understanding of the codebase and where things are.
The model that does make sense to me is (and the only actual success stories I've seen) is people saying "it let me quickly produce a piece of software that otherwise wouldn't have been worth the time to create". That is definitely an increase in productivity, but "software people aren't actually willing to pay for can now be made much more cheaply" is a much different claim than the marketing is making (which I read to be TFA's point).
I don't really see the influence of things like LLMs (or StackOverflow or improved search engines) as simple productivity. People do what they can with very complex value estimates and comfort levels. If they are less productive in a careful measure it may mean they are doing a lot of high value low hanging fruit across areas they were afraid to touch.
The trouble with highly productive specialists is that they produce a ton of high quality results where the demand is not really there and has to be artificially made. Even if you find enough work for them it often means the incremental cases are things you wouldn't have bothered with. A specialist branching to work slowly in related yet further related areas is a lot more value and can work with an oracle so flawed that it barely beats chance..
With juniors it is much more complex, but they have always been a useless consideration in productivity. Not having them has always been highly productive in the short term but has long term consequences.