Comment by torben-friis
3 months ago
>A caveat: Lines of code is an imperfect measure, as it measures quantity over quality. So 8× lines of code/engineer/day in the second quarter of 2026 is almost certainly an overstatement of the true productivity gain. Nonetheless, it indicates an acceleration. At Anthropic, we don’t reward people for how many lines of code they write; rather, team members are producing more code simply because they’re using AI systems to write more code.
What about the hypothesis that AI is generating more verbose code? I just see the text pretending to acknowledge "LOC != Productivity" and then using it as a metric anyway.
One of my co-workers just asked me to review his pull request that was all AI generated. 600 files were touched, over 40k lines of code added.
I'm sure he thought that was a crowning achievement, proof that AI can enable 10X developers, after all, what engineer could write 40k lines of code in a week?
I declined to review it, stating that I couldn't possibly vet 40k lines of code, and wouldn't put my reputation on the line to stamp the work as good. The PR nagged me for 2 weeks from my todo list and then disappeared. I don't know if he found another dev to get an approval from, or if the PR was abandoned. But I know for sure that him and I are on two totally separate islands around the value of LLMs.
Same here. A co-worker touched a few hundred files in a PR and asked us to review. They merged it directly to main when nobody approved it. (The repo was not set up to enforce PR approval.)
I don't personally use that feature, and I couldn't care less at this point. If our customers are frustrated by the bugs, at least my name is not on it.
The challenge is that you may not have customers, and thus a job, if that continues
1 reply →
Crazy they merged into main holy moly
That's a process problem at your company - no developer should be proposing branches over 1k loc (or whatever your agreed tolerance threshold is) without a very good reason, vibe coded or not.
It isn't about small or big, it's about cohesion of the changes.
I prefer a big feature to be one big PR rather than a lot of small ones.
We had a dev do a big feature with a ton of small PRs, each one was individually impossible to review because each concern was out of scope for the small PR and "would be fixed in later PRs". Once it all came together as as whole, the big picture was a total horror show and I had to rewrite basically the whole thing.
In order to review those small PRs properly, each time I would have to read and understand all the current code so far from the beginning. Without that, each small PR individually looks OK because you won't remember the other PRs from weeks back that already duplicated what the current small PR does for example.
2 replies →
> no developer should be proposing branches over 1k loc
I've seen that reaction many times. It seems to work well enough when someone is maintaining existing code. However, greenfield projects can often require literally orders of magnitude more code to deliver something that can be integration tested.
The first step is to break it up into a stack of commits. Each one must compile and pass its unit tests, of course. Keeping it under 1k loc of released executable code is usually easy, but often becomes difficult to impossible if you want well commented code with excellent unit test coverage.
Assuming you have kept all your commits under 1k loc, there is still the problem of whether you present them in one PR, or as a stack of PRs. The issue with a stack is why an API is designed a certain way often isn't evident until you see how it's used. Responses to PR comments are explanations that point to later PRs in the stack, which is irritating for both the reviewer and the author.
I haven't found a good solution. I'm not sure there is one.
1 reply →
> no developer should be proposing branches over 1k loc
I completely agree with you. But I am afraid we are losing the battle.
I am seeing people repeatedly sending out gigantic PRs full of slop, code with mistakes that they would never have made if they were hand coding it. And they don't care. It's sometimes surprising if not horrifying to find that the colleagues you have worked with for years don't care about quality at all -- almost despising spending time reviewing their own code. Yet they have the audacity to send out code reviews.
A former coworker sent me an AI generated PR to review and I just said NAK after the first two issues I found and I said to not send me AI slop to review.
They went to HR who said I am more senior and I should act as a mentor (they had my same work title and were probably making 4x more due to being in USA) and I just no longer reviewed anything from them until I changed jobs.
People actually go to HR for such trivial things?
1 reply →
I declined to review it, stating that I couldn't possibly vet 40k lines of code
Gee, that sounds like a job for Claude if there ever was one.
You're absolutely right!
And how would you verify that the review was accurate?
8 replies →
At work we had copilot. It said "the diff is too big to review"
This is a branching point. One dev would find someone else and convince them to approve it. Another would redo the task (code is cheap now, right?) in a PR stack that can actually be reviewed, cleaned up etc.
I hope they were the latter.
My review would have been along the lines of:
'Please split this PR into smaller ones'. I would even sketch which groups/phases would make sense, perhaps with the help of AI.
You could surely check on the status of that PR.
I just watched copilot today turn a 8 line fix into 500 lines, so, yeah, verbosity is a big side effect
It occurs to me this pattern might be the average code we humans have produced. We all have made those quick fixes, copy-pastas, and dirty hacks... they learned it somewhere! I also assume that some of the behavior is an artifact of their training regime.
So with LLM outputting average code, and people using LLM more and more, I guess the average code will become worse over time ?
12 replies →
In my case, where I see it most often is when the LLM has to rework something multiple times, and the feedback loop is vague (especially when all I have to give it is "no error messages, but it's still broken"). It seems like after the third or fourth try it just kinda goes off the rails. I find that the one-shot quality tends to be a little better, if the slot machine happened to work correctly that time.
3 replies →
If you can make it 800 you can claim to be a 100x engineer!
Missed opportunity! I obviously have skill issues.
Real question is, are you a 100x prompter?
So the more rigorous studies about AI-assisted coding productivity addressed this by keeping in place all other software development processes, including the same code review and quality standards, and only measuring throughput (PRs, LoC) before and after AI was allowed.
Hence the intepretation of this 8x number depends on whether (or how much) Anthropic engineers have changed their quality standards and development processes. They don't tell us, and I am not aware of any other indications we could use to make a judgment.
However, we can still do some theorycrafting! I'm convinced that to fully realize the potential of AI-assisted coding we need to revamp all the dev processes, especially how we validate code, and it would be foolish of Anthropic not to do so (unless they were conducting a rigorous study, which they don't claim to have done.)
My hypothesis on the future of software validation is nothing fancy, we simply want much, much more automation for tests, observability and other bespoke verification methods than we traditionally had. But then validation code will also contribute to the LoC! My observation so far of personal as well as some "vibe-coded" open-source projects is O(LoC production code) ~= O(LoC test code). So as a SWAG the upper bound could be something like a 3 - 4x speedup, which is still remarkable.
All bets are off if code quality standards are not the same.
Exactly. If AI is going to start being graded on how many LoC it generates- oh, I'm sorry, how much it "accelerates", than guess what newer models will start doing more of?
Surely they can train AI on the signal to change as few lines as possible. Indeed, this is something I'd want to have control over when making requests. In a traditional UI, I'd imagine some kind of slider between "fewest lines" and "be bold".
I've been having some success asking Claude to run sloccount after each change. Seems to help a little, though it's prone to forgetting over a long session.
I'm actually hopeful that the recursive code training will improve quality over time. I'm definitely producing higher quality code, tests, and docs. It does take attention and oversight, iteration and refinement, one cannot just let these things loose on a code base and expect good things to happen. You have to leverage them to make the good things happen.
I don’t understand how lines of code matter at all for scary LLM core capabilities. Does the transformer architecture get better with more lines of code?
My impression was that LLM training codebases were 99% resource management and only a few lines actually implement the core training algorithm, which is where 100% of the intelligence comes from. Data, not lines of code, are the constraint.
After training you can adapt the intelligence in various ways, and that takes a bunch of lines of coded too. But you cant raise the intelligence ceiling again without another training run. So where is the scary recursive part?
Yeah, they assume that "productivity = k * LOC" where k > 1
very flawed
Could just be more tests? :) Which is good for code quality in general and reduces support burden, but doesn’t lead directly to more features
Just imagine the productivity gains from using LLMs to rewrite Kotlin codebases in Java!
AI generates code that mimics the existing code. If your code is terse and comment-free, then the agent’s code is too. The times I’ve seen Claude drift into a default “house style” it generated like 1 comment for every 10 LOC or so. It’s a far cry from the GPT-3 days that littered every line with the journals of Captain Obvious.
That is definitely not my experience using Claude Code with Opus. I work in a very sparsely commented code base, and the agent produces substantially more comments than the surrounding code.