This is like one of Google’s infrastructure blog. Google scale things are only relevant and appropriate for them. We had lots of bitter lessons from smaller engineering teams following the process with the justification that “this is how Google does it”.
Similarly, Anthropic can afford to do this because they have virtually unlimited tokens and labs to train models for their specific usecase. The rest of us need code comprehension lest we become dependent on continuous supply of tokens (which is a state these labs badly want to achieve)
So, unless you work at a lab, or have unlimited tokens guaranteed through the life of your company, it would do both the company and you good to continue to read the code and not let these 30k PRs per engineer be the norm.
An excerpt from the article makes my point
> Nobody on the team keeps a complete mental model of Claude Code in their head, and we don't think anyone needs to. When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today.
Even then do we really have evidence that what they are doing is reasonable or practical for them at all? They are edging on like 3 9s uptime, what leaked of Claude code looked like soup, tons of security issues, they can't build a sandbox to save their company, and last I heard they aren't even profitable without adjusting their numbers for arbitrary things.
Think I'm good on taking advice from companies that aren't making money.
Exactly. What they’re doing, repeatedly, is setting a narrative that this is how software engineering should be. Always be reliant on tokens. Don’t worry about uptime or errors - agents can address it, don’t worry about security - agents can address it, don’t worry about code comprehension - agents will always be there for you.
If they're cash-flow positive with 75% of compute going to research and training, what do you think will happen to cash flow when they shift to 50-50 training-inference?
Because "pacing the frontier" is more about flipping on the money printer switch than it is about safety.
> they ask Claude to write an explainer for exactly that slice
If their Claude is the same as our Claude, then this is a joke. Claude’s writing style is insidiously mind numbing. It takes constant pushing to get it to explain things in digestible English.
It also mixes important with unimportant information. It tends to think that a commit/bugfix is important and will happily list it together with important properties of the feature.
At least Opus 5.5 does that. Its writing is better than that if other models, though. I had other models explain stuff in an undigestable language, technically correct, but practically not what anything is about.
Try having it explain something in “ISO Plain Language standard”. It made an enormous difference for me yesterday. The language will change from dense large paragraphs to simply-worded 2-3 sentence paragraphs. I was using gpt though.
Even getting it to write less obtuse English isn't enough IMO. It seems to have real issues with structuring documentation in a way that makes it easy to follow. Also it's a real demon for leaking its context as implicit assumptions into documentation.
They pretty much fixed that. Plus you can influence the writing style to some extent. I tell it to use full sentences and avoid assuming I already understand the code.
Another thing is that it's very good at creating diagrams (I think someone wrote a skill for it so maybe not by default). I usually find the diagrams are even more valuable than the text.
Honestly I was skeptical but you really can just point Claude at some code and ask it to make a markdown + SVG explainer and it will do a shockingly good job.
I'd still rather read human documentation, but even before AI almost nobody writes any - especially for internal code. And they certainly don't make great diagrams.
If we as engineers didn't use similar terminology to describe any code not written by ourselves for the last few decades perhaps people would take it seriously.
I have heard from someone who works at Anthropic that they will have postmortems that sometimes devolve into "my claude says X" vs "well my claude says Y". Maybe this is the future of software engineering, but it seems pretty fucking sad.
This is basically just being a manager in the before times. "My engineer said X!" "Well, mine said Y!"
Which is fine, whatever, I guess this is how I pay the mortgage off instead of having any clue what the hell Claude is actually writing. Hopefully I can semi-retire soon and go study dunnarts or take up welding or something else that's more rewarding than this nonsense.
I would love to ask them why this scale is even necessary for a conceptually simple CLI app like Claude Code, but they wouldn't be able to answer because they don't even understand the scope of it apparently.
They do have quite a few products now, the desktop apps, websites, cowork, cloud session sandboxes, API, enterprise, etc. have their own gotchas, but after a core development, the 30000 commits a day does not feel justified, or is that an online training dataset prep, that's hard to tell. That's why I feel with their endless tokens, it feels just petty marketing and that's sad because it does not bring much to our engineering table.
It's absolutely insane that they can't keep track of a CLI application that makes shell calls. This is so far from rocket science even mentioning the term is misleading.
It needs to gather a bunch of inputs and feed into the remote GPUs correctly and as efficiently as possible. Lots of room to optimize under the constraints that users love cheap tokens, demand a fair billing structure, and want responsiveness and low latency.
I don’t think you can infer the scale for Claude Code alone. They said it doubled but not what the starting point was. The 30k figure is for the entire Anthropic monorepo.
> Nobody on the team keeps a complete mental model of Claude Code in their head.
For sufficiently large codebases in large organization that is how it has been with humans, who come and go. In many respects, What Anthropic is doing is a massive upgrade from the status quo.
Another challenge is getting developers to stop assuming they know what the code does without reading it! Having AI “fact check” various documents against the source will help immensely.
Really? Hundreds of billions of dollars in debt is an improvement? The product that this code is supporting can't even pay for its debt.
Jesus. Give me 1 billion dollars and I'll keep your entire codebase in my head + personal notes for you until I die. Hell, I'll even force my descendants to do the same. Now _that's_ an improvement.
It would benefit many people (especially Anthropic employees) if, after an LLM gives you a response like this, to go into a different session and ask another LLM if the explainer is correct. Not just so that you hopefully get the correct information, but also to see how easily fooled you can be by a plausible-looking, confident response about some code that you have no reference for. Use 5.5 on any thinking level, you will encounter this. The spaghetti mess that is CC likely makes this more of an issue.
Google's infra genuinely pushed the industry forward though, things like kubernetes and SRE methodology are generally useful even if you aren't Google. Reading this I'm not even sure this sounds like the best way to run an organisation with infinite free LLM tokens let alone an actual engineering org where it just an impact on development cost. The people who are getting maximum leverage from LLM reasoning and routing queries intelligently who've made big strides in reducing inference while increasing throughout are the ones who impress me the most.
I saw someone say the other day (in response to another blog post being well written) that Anthropic employees have a rule against using AI for writing/external comms? That seems clearly not the case here, normal people don't write like this:
> Most of the path is automated; a person's attention goes where the risk is
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
100%, although if people are getting 90% of their interaction from a robot, one may start to pick up some vocal tics and colloquialisms from machine world. a kind of robo-pidgin.
English isn't my first language, and I'm treating every opportunity to use the language as a way to learn, including extensive discussions with AI at everyday work. Sadly, I'm not noticing any unnatural wording in its replies.
I'm starting to worry about being gradually affected by AI's style through heavy use. It will make me so sad if one day my written piece sent to others is considered to be AI-generated, while it's written with real effort trying to make it sound natural.
> That seems clearly not the case here, normal people don't write like this:
I think when you repeatedly expose and immerse yourself to such a degree in reading and writing language, it brushes off on oneself. So I could also absolutely believe someone from Anthropic wrote this.
agree 100 percent, I write prompts, and at some point I'm going to adjust my style when I see my prompt is ambiguous or something and I bet subconciously I'm going to try to talk like my robot pal when making these adjustments so I could see it happening
this definitely seemed like something claude generated, i know because i frequently do all my research in chat and then ask it to generate a doc based on what we talked about and when i read what it generates the verbosity, the style it writes in makes me delete the whole thing. I would never subject readers to that.
It felt weird reading it because every once in awhile there were some well written sentences and coherent paragraphs, then it fell back into Claudisms.
As a non-native, I see how susceptible I am to absorbing such style. There's no way at all for me to perceive anything wrong with the examples you present
It's a bit more subtle than things like "it's not X, it's Y". And even that isn't "wrong", it's just the way AI talks, and therefore an indicator that the author offloaded their thinking to an agent. I've read so much Claude output that it's easy for me to recognize its voice, the same way I could recognize a friend's voice in writing. The examples I quoted are a weird kind of circuitous sentence structure with a certain shape and rhythm that scream Claude to me. Here is how I might rewrite them in human speak:
> Most of the path is automated; a person's attention goes where the risk is
--> We automate as much as possible, focusing human attention on risky parts
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
--> We run reviewer evals against every incident
But that's just my interpretation and my voice. I can't know for sure what the authors would have said if they took the time to write it themselves.
Personally, a year or two ago writing like that did not really come across as particular bad or AI generated to me, although it was uncommon for humans to write phrases like that. Writing in an unusual way is not inherently bad, it can make your text interesting... but it is less direct and clear than alternatives. But the style becomes unpleasant if you keep seeing the same patterns over and over such as "X'ed against all of it"
What came first, "Claudish" or those who trained it? I wonder if Anthropic's employees originally wrote like that. It's definitely irritating to read, and IMHO more so than the ESL-grade English which was quite common before AI.
Claudeslop didn’t come from humans, it’s an artifact of running RL on slop outputs. If you keep feeding the slop output back into the model and then training on it, it amplifies its own unique weird vocabulary and behaviours.
No wonder most people at Anthropic believe our jobs are in jeopardy, they've already made a system wherein the machines are treating them like pets.
I really don't get the tendency here from Anthropic to describe what's most likely churn? A high pressure amount of changes/PRs a day on a TUI to wrap a while loop? I'm not going to be a reductionist here so I'm going to cut it off there, I'm positive that the complexity of handling multiple terminal emulators is something that needs some handling (even if doing their own rendering engine was a self made problem). Yes I've vibe coded my own harness from scratch without writing any of it, I still know how it works.
I'm also not sure that they addressed point 2 in any actual capacity? They say "the system verifies itself" which is cool and is something that automated testing has done for decades, but they eschew the mental model of how the system works as a whole and prefer instead to have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
N.B. Changed "1000 PRs" to "A high pressure amount" to be vague since the article is vague here on what CC undergoes.
> have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
This is the crux. Take pause, because hidden here is a sea-change in our fundamental trust model of software...
There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug. By design, the system is too large for that, and everything hinges on Claude itself being able to find and understand the right slice of the system in any situation. As long as that very new invariant holds, everything will be all right. If it fails catastrophically, there is no recovery. You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
The new trust model is: We trust Claude to always understand the system enough to debug anything or make any reasonable change we could want. The risk might be worth the reward. But it's worth explicitly laying out the deal we're making.
Tbf, it's worth pointing out that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
"Tbf, it's worth pointing that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
The real difference as I see it is that in the human maintained old style system, the human would take things slowly and ensure they understand any changes. Claude let loose will just say "yes, I know how to do this" and bork the system, because it doesn't have a concept of humbleness in the face of complexity!
> You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
I love this phrase, and I think it captures the difference between the "60 years of legacy code" mentioned elsewhere here, and the vibe-coded monstrosity that we all know we're going to be encountering in the next 5 years. The fact that there were human brains with intent making all the little (short-sighted, myopic, ignorant, etc) changes in the legacy code means that you DO have at least some shot of gaining enough context to make safer changes.
There is no real intent in LLMs and particularly in blind vibe coding, so you'll end up reading stuff that works, or appears to work, but there was no human brain behind the change for you to model and thus predict.
most large software companies are already too large for anyone to have a complete mental model of.
The idea that someone at say, Google, will be able to fit even half of the system architecture in their heads is not realistic. It also hinges on being able to find and understand the right slice of the system in any situation.
But we have all sorts of guardrails to make that easy - i.e logs, traces, etc
> There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug.
At this point, why even bother with source code? Source code is designed for humans to comprehend. If humans are no longer the intended audience, why not have machines output and maintain some more efficient intermediate representation of computer programs?
I fixed many bugs over the last years with help of claude code in an unfamiliar codebase. it is never just Claude alone so we don’t rely on AI understanding 100%. It is whatever AI can understand plus what I do. If you use AI right your understanding will also grow over time.
Thank you! This is a huge sticking point for me. There is no knowing the code without experiencing the code. If nothing else, you have to read the tests to know they verify what you think they do. Even then, you might know it does what you want, but can’t know it doesn’t do anything it shouldn’t.
> wherein the machines are treating them like pets
I know some people will groan hearing the phrase again, but: "Reverse centaur." A centaur is a synergistic combination of two things, but now those things are in the wrong position: The human is the computer's tool.
Naturally, how "wrong" it is tends to depend on whether you ask the people who do all the work or the people who own all the computer-capital.
When you get to the point that you are doing 1000 PRs I have to seriously wonder what even are you building, I hear about this a lot from a number of companies, but their “too many commits / too many PRs” problem never seems to give anyone from the outside the feeling they are innovating at groundbreaking speeds.
My favourite quote at the moment: "May not man himself become a sort of parasite upon the machines? An affectionate machine-tickling aphid?" Samuel Butler, Erewhon, 1872
I think it's just a different focus on what matters.
There's clearly enormous (human) work that goes in to building and tuning their models.
The software they develop around their model is, relatively, trivial. It's not clear that enormous amounts of (human) energy need to go in to that code.
It seems largely a function of applying human energy towards the things that are marginally most important to generating you money and putting you in front of competitors.
I mean, sure we can assume that (although the Anthropic article from the other day say that claude mostly builds Claude) but this is the Claude Code team which, I again am assuming so I could be wrong, focuses on the TUI/CLI interface to those models, not necessarily the models themselves. That coupled with the fact that this is how a large portion of their clientele interact with their moat, it would be odd to me if they didn't apply significant work towards this thing to keep the hype or productivity train moving
The Anthropic post the other day mentioned part of what helps Claude Code unlock some of this is the sheer ability to use more tokens than the rest to get something going and efficient.
> No wonder most people at Anthropic believe our jobs are in jeopardy, they've already made a system wherein the machines are treating them like pets.
I guess it wouldn't be surprising for people prone to AI psychosis to be attracted to working at the frontier labs, resulting companies that are heavily in the throes of mass AI psychosis.
The ownership component is the toughest and least enjoyable part of coding with ai. AI saves the time of coding, but 100% of the responsibility remains on you. You then have to invest an imbalanced amount of time to understand and fix the issue vs the time it took to generate the code. Then on top of that is the emotional feeling of something you “owned” messed up, but it wasn’t really your doing - it was the AI that did the keyboard work and now I need to parent/manage the AI. This destroys trust in AI and the cycle restarts. But we keep coming back to the slot machine as we often get those returns where things just work, but like gambling, you’re not satisfied by those wins and you’re immediately on to looking for the next fleeting high.
I think the question is then "how do I gain more confidence this code is correct?" This doesn't have to be reading through everything line by line, it can be writing more tests, or working with the LLM to dig into how certain edge cases are handled etc. There are a lot of things you can do to improve your confidence in a piece of code.
> You then have to invest an imbalanced amount of time to understand and fix the issue vs the time it took to generate the code.
Obviously no amount of training will make AI human but: how is this different from you being the leader of a team whose output you own and are accountable for?
The difference is that now people who signed up and were hired as engineers now have to do what managers used to do. It's also a different pay grade when managers are accountable for a team vs and engineer for Claude instances.
In a weird way, it reminds me of playing chess. If I play a human opponent I can be thoughtful and deliberate. If I switch to playing a bot, I respond to the speed of the responses no matter the level and get crushed.
It takes a while to settle in and play deliberately. I also need to switch gears because bots play much differenly than human opponents - they don't fall for bait, they don't have time pressure at all, they don't leave pieces uncovered by accident.
I think of coding with AI is similar, you need to have a framework where you build understanding along the way from plan to review. It is untenable to roll out a 6000 line feature then try to review it, even if the LLM is happy to do things that way, mere mortals can't keep up. But if you stay in your lane and code solidly, AI can be a straight speed boost. I feel like most people despairing over handing the reins to AI simply haven't found their personal sweet spot.
I look at it as a giant experiment. Any research tends to be very investment heavy, and people are given a lot more leeway to try things which are wildly unprofitable.
I'm glad they're doing this, because it lets me see an extreme of LLM development.
We spend a few thousand dollars a month in AI code review costs.
Compare that to the cost of human code review which can take up a third of an engineer’s time. You’re looking at $50-100K a year per engineer fully loaded in code review costs.
If AI assists in code review (not even replaces – we still do human review ourselves), you could spend thousands a month per engineer and still come out ahead.
Ironic that this comment talks about cost without weighing the numbers.
It’s gotten pretty good too. My employer has a team that wrote their own code review bot with the checks and values we care about and it has gotten quite good over the last six months.
"Anthropic told investors that it will profit on a measure called adjusted operating income.
"According to the report, adjusted operating income excludes stock-based compensation, which is not unusual for technology companies. However, the report also quotes its gross margins - above 80% - exclude revenue shared with distribution partners, as well as the cost of training its models."
Antropic's revenue is up [1], [2], [3] and if you calculate adjusted operating income (by excluding most expenses like training costs) then by that measure it is profitable. However, if you factor in all the numbers it is operating at a loss.
Source? Only reports I’ve seen was they were profitable for one month in last quarter due to xAI discount of renting out colossus . Be interested to learn otherwise.
Anthropic has amazing models. But they should really stop commenting on software practices. Most of what they say seem out of touch with what's happening in software shops. A lot of it seems hype driven. And the entire message looks to be written by AI.
> But they should really stop commenting on software practices.
I disagree. Many of us have an emotional attachment to the way we do things and have difficulty letting go. Anthropic is showing a different way of doing things. And I am not going to judge them too harshly if the final output is good. I use Claude Code every day. And it is good. Ahead of the pack.
You're failing to differentiate what Claude Code means here. The models, no one argues that they're not good. If you mean Claude code the local agent, there are now dozens of smaller, faster, lighter harnesses that do everything the Claude tui does and more.
Yes, this is a different way of doing things, and it's clearly working for them, but there is no direct line between that and a "ahead of the pack" product.
I don't understand saying you have an emotional attachment to doing something. Normally you do things in a certain way because it is functional and works. What we're seeing from these LLM companies is them proposing new methods of working which highlights insane ways of "working" (they don't "work") since they're reliant on an LLM, which is just self-marketing.
Given the internal models that they have access to, I'm more worried about software companies being out of touch with the next generation of models will enable. Anecdotally I've seen that it takes a while for an organization to adjust to new LLM capabilities, even if everyone had access.
It’s all marketing nonsense. Along with the “AI is dangerous” and “we’re all going to lose our jobs” messaging they do, it’s just to sell the services.
From the article: "merge with at most one human in the loop, who can be the author". Curious to see how this handles enterprise compliance requirements and insider threats?
- How was this approved for SOC2 compliance? Our SOC2 reviewer explicitly mentioned that all code needs to reviewed by at least one other human before it goes to production. Claude Code says they are SOC2 compliant (https://code.claude.com/docs/en/security).
- This assumes all engineers will not do anything malicious. What is preventing a Claude engineer from spying on their ex's or some other companies Claude searches by pushing malicious code? Recently an OpenAI engineer was caught planting a recording device at their own office.
I’m sure it’s not compliant. We have the same requirement for soc2. However the audits are pretty light and a company big enough can afford to have an auditor who looks past those things.
i think SOC2 is about audibility more than anything - my guess - and i’m not an internal auditor or anything - is soc2 needs auditing for human changes, but a machine is just a “system behaving on its own”
No human can stay on par with 30k pull requests per day ... they simply don't have control on what's going on and rely on AI itself to do the job.
PS: I really want to know what is the churn rate (and the cost per LOC) here. My gut is saying the cost per LOC is insane, and they are basically rewriting over and over the same code.
30k PRs is such a meaningless figure though, you can't make anything of it. It entirely depends what's living in that repo. Adding localized strings, configs, things that change frequently, can easily move numbers like this by orders of magnitude. How big are the PRs? Maybe there's an auto-bug-fixer agent that runs around the codebase making thousands of single-line fixes per day. Who knows. The number means nothing in isolation. We used to scoff at bad managers for measuring the performance of engineers using lines of code, now we're using the same figures to judge AI. We've become the bad managers.
Actually LOC is a decent measurement, it becomes a bad measurement when you use it in the wrong way, like any other measurement.
Agree that 30k PR alone doesn't mean much, but there aren't other details on what's inside. Based on I saw on how LLM programming works, it's hard to believe that there are single-line fixes. If I have to bet I'd bet on 200 LOC on average, that is 6M/day. Even if only a small portion reach production, it's still insane.
They aren’t claiming that the doing less by itself changes the rate in this sentence. They are saying that the rate changes when you slow down, probably due to having more time to focus, which in my opinion is likely.
Haha, yeah.. Also with most real world, vs LLM FOMO hype fantasy land, consumer software has two very real tiers of bugs; regressions, and bugs in new features.
Regressions are usually the absolute worst.
Up to now we've just been eating shit on the DX side as far as regressions in Codex and Claude Code because everything is all "go-go-go OMG I don't want to fall behind" and also.. Access to new models is tied to harness updates.
But a ton of industries have users that are not so willing to eat the regressions in the name of more features and changes than they can keep up with.
Edit: Also moving too fast.. You may have been less likely to "get it right" than ever before leading to design churn. Again, depending on industry and users, they may not be very tolerant to releasing the 5th complete overhaul of some feature in so many weeks..
How is the ability of a human to understand the code via hands-on familiarity not a direct dependency of their ability to do manual code review? I don't get it. Having claude write an explainer (something I do a lot!) doesn't get you to the point where you can confidently assess a complex diff. And you've funneled your priors on what the diff says and does through the same system that already reviewed and approved it. This article isn't bad but it doesn't give enough weight to the human context-building part of the problem.
I think there is no magic trick here. The answer is simply that humans will no longer understand the code.
Although, following Anthropic’s ethos and taking this to its conclusion, I’m not even sure why they need human reviewers at all. What, they bring in a human to look at a diff of an “important” section of code that was written by agents and has changed a dozen times since last week through 12 PRs of churn? Obviously they don’t know what’s going on. So what’s a human to do? Do what the other approvers (if any) did: fire up a Claude to command it “verify this change has no regression; check carefully for edge cases”, then come back after a half hour of High Thinking to see “Change is clean (with some non-blocking findings below, in order of significance)” and click Approve. Not sure what the purpose of the human is in such a process.
Fair assumption. I think humans, at least those who like to code and build solutions, will become better builders than coders. Understanding the code will become less important, but knowing how to put it all together will be essential.
Yes it used to be said, the code is not the asset, the people who know what the code does and how to change it are the asset.
Now we’re just going to outsource our capacity to design and review because it’s faster that way, as if the thing that was wrong with the software industry was that releases weren’t being pushed out often enough.
I agree with you. A code reviewer from the same company, even with different models or efforts, shares the same blind spots. To review my code, I use a different company; if I code with Claude (usually Sonnet 5.5-medium), Codex reviews it, and vice versa. I get much better code this way. But more importantly, I always tell the reviewer to build its own test-scenario ledger and refute the coder's claim that it is done. And the verdict is shared with the coder, but the coder cannot reject it.
I'm thinking about high volume mass manufacturing as an example here. I like those "How it is made" type YouTube videos. You see a production line that has some kind of QA process where cameras + automated systems do spot checks at high speeds and these crazy actuators are punching out bad examples. These processes often also involve spot checks where some statistical sample is selected and viewed carefully.
I don't think we've gotten to that level of reliable review with code, but I don't see any reason we can't continue moving in that direction.
I really like those videos too, but a very big difference is, in manufacturing QA, the system outputs a very high number of copies of a very small number of possible types, so spot checks work fine from an operations research point of view, as the variance in output has a known upper bound.
For code, I think things go in the opposite direction: the system outputs artifacts that are all different among themselves (even more so now that we no longer distribute software via physical copies. Well, bits are always physical, but I mean we no longer ship floppies/CDs/DVDs).
That said, at least in my experience, code review has never been a direct part of factory-line-type QA. That QA is typically done by treating software as a black box and just verifying that it complies with a spec (often times based on a very loose definition of "spec" ...), and this is something that works equally well regardless of whether a human or a machine wrote the software. The type of QA that is done (among other things) by code reviews is checking whether the software's design is such that it will be economically feasible to maintain and extend it, for example.
The one part the code reviews takes a part of (but is in no way error-free) and that you just can't do with black-box testing (or it is much harder to do) is to look for things like backdoors. As our industry moves more towards "agents write software and tests validate that it works," I think a lot about "Reflections on Trusting Trust" and the new and wide attack surface added by people no longer reading the code shipped on their behalf.
That’s a category error. Software isn’t stamping out the same part again and again; if it were review would be simple. Analogies often mislead.
Of course Anthropic would love you to think you are heading towards that future so we have to bear in mind their livelihood depends on LLMs being the future of work and 30k changes a day being normal.
But why would we want 30k changes a day in a given product, what possible justification could there be?
Those manufacturing videos are of someone making the exact same thing over and over. Months and years of testing go into ensuring that there is no change in the system, and continuous monitoring is put in place so that if any changes slip in, it can be undone. The design is set in stone, as it should be, and variations are bugs. There are months to years of cheap prototypes, iteration, design changes, pipeline tuning, and verification to ensure that nothing will need to change for the next million items.
If you want a software analogy to those devices getting churned off the assembly lines, it's the TCP streams coming out of your network card, with crazy checksums punching out bad packets.
We're orders of magnitude better at reliability with code, but we're regressing on that metric rather quickly. And the comment above deeply misunderstands both manufacturing and software.
Those machines are not doing design work. They automate production of designs that had long development and testing lifecycles. Code is design. The compiler is the automated production process.
Why not? Code review is a crazy time sync. If I'm working on a project without code review (e.g. a personal tool) using Claude I can easily churn out 100 commits per day. And it isn't even slop these days.
It does make me wonder what on earth they are working on. Feels like most of their products could be basically finished already.
On the other hand Spotify somehow employs 3000 programmers, and that's just a music player.
> We don't have a formal process for this. Nobody on the team keeps a complete mental model of Claude Code in their head, and we don't think anyone needs to.
Nobody on the team keeps the mental model of Claude Code as it is today in their head, so nobody on the team is able to reason and decide about Claude Code in the future, is that right? So no long term architecture strategy can exist as no current architecture understanding exists. While it seemingly does work for a system which is a TUI, would the same be true for a complex distributed system today?
'Nobody has a complete mental model of <insert name of system>' has very much been the norm since the invention of the two week's notice. What's actually new is that an architecture strategy can now be cheaply reverse engineered out of any system, adjusted where needed and immediately applied to the development process in days or hours since 'the team' is now a single dude or dudette with a fleet of robots which can see the whole code base together with its durable arch docs at once.
Before all came to this, I've been promoting reviews at $dayjob not as gatekeeping but as method to exchange information and build more team spirit. We are an internal service team working by "projects", which makes us a very disjointed team with barely any communication. (Part of it is also German work culture.)
And now this. Suffice to say, my efforts did not bear fruit. I feel more isolated than ever before.
Reviews were always meant to be about more than just finding bugs, thanks for mentioning that.
Nowadays I'll ask a question to, for example, try to understand why was a particular abstraction chosen, and I'll get a response from a robot, sometimes without even a disclaimer that the response came from a robot (it's still somewhat easy to spot due to the shape and style of the prose).
It does make me feel like there is nobody on the other side, especially working remotely, so I think I relate to your feeling of isolation.
Sometimes I care about how some change impacts the broader application moving forward, not really the specific technical tradeoffs for one construct or another, and that's, more often than not, very lacking.
Does anyone ever read what code these models produce? 50% or more is just crap: meaningless tests, convoluted code, a lot of indirection, unreadable with a lot of bugs. I am not surprised that they merge as much as they do, they just dont know what the code does but see it is somewhat working. Most people who are impressed by it, I am sure, are just shit coders themselves ngl
> Most people who are impressed by it, I am sure, are just shit coders themselves
The less you understand code, the more you're impressed by coding chatbots - in part because you're less likely to read the bot's code, and if you do, less likely to realise how poor it is. Code gets judged good simply because it exists and costed little to create.
It's funny to me that they use Claude Code as advertisement for their coding models. This software is mediocre at best, full of useless bloat and really only good enough for the use case (unless, the use case is advertising their models to vibe coders, then it's pretty good and I'm not the target audience)
That's low risk when the flag is off. Is there suddenly proper code review before the flag gets turned on? That's when the risk is actualised.
Also, when they say "risk" - risk of what? Bugs and security incidents? What about other risks: future rework, slowdown of development, obfuscation of bugs, abstractions that will end up being made leaky to work etc. Are these all not considered because they are confident Claude can manage all of that?
I don't know if this is just the Claude Code team that works on the Agentic harness or the one that does the backend model hosting. But if it's the former, I think posts like this are exhausting because their stack is so simple that a setup like this is possible.
I would love to create agentic feedback loops like this but my project (and probably a lot in the software industry) are compiled. Compiling not only takes a lot of time, but we have to carry around nearly 200 GBs in toolchain dependencies, require access to license servers for 3rd party software used in the toolchain and to run the full integration test suite on the project can take hours even with parallel testing in the VM while using multiple VMs. And these VMs aren't small, 64 core/128GB. So outside of unit tests, we can't really merge code fast and test it with high confidence without spending an absolute absurd amount on VMs in order to enable the agents to move quickly.
This was an exhausting post. both the writing style and the thought that this is what bleeding edge currently looks like.
Some of the most capable engineers in the world high on their own supply… I have no doubt they think this is aspirational but even as I know LLMs will forever change how we code, I’m not entirely sure 30,000 PRs for a coding agent is a volume that makes any sense.
All good ideas, I have been doing something similar for half a year. The most important thing is to reduce bugs and ensure the code does what you wanted. Writing code tests, logging, interacting with the running app are essential. For intent realignment I like to have the agent re-read all user messages posted in that project. It only takes a few minutes even on a 10,000 message project. Then use this fresh derivation of user intent to check code and tests. For long spans of work I prefer to generate a todo list which remains as a document in the project. Each closed gate tracks feedback from the agent on what happened. This task file is also reviewed twice - once before implementation and once after. So the formula is - review agents - tests - user message logs.
You don't need to read my message if you think you know what is the best formula for your AI usage. Whenever someone complains to be "tired" of reading some online message I take it as an act of verbal aggression. I rarely disclosed my formula online, and probably never will again after such reaction. Do you want to regulate HN? Was my message off?
Staff software engineers(because all the seniors and below have already been sent to work at the soylent green factories) will have AI monitoring software installed on their computers.
If it detects they aren't paying attention, fall asleep, or haven't approved enough PRs per hour; off the work at the factory.
I snorted laughing at this. "We are experiencing tremendous growth" and it's just a codebase with 1,000 agents endlessly adding markdown documentation for every 1 line change.
I just learned that manually invoked skills MUST be the _first_ thing in the prompt, e.g. you can't have `disable-model-invocation: true` and say "Hi, /grill-me on X".
Worse, everything after the /grill-me is injected a second time into the prompt as a <command-args> so your prompt is literally added to the context TWICE. (first time in the original user prompt.)
I swear Anthropic is intentionally bloating your context so you burn tokens on useless bullshit like this.
Codex had over 15k issues last I checked including legit months old issues with the sandboxing/bubblewrap config.
Human contributions to issues, comments or submissions, were drowning in a sea of AI vomit. Tons of the issues had those AI issue submissions with the classic confidence of having identified the problem, but that smell I know a lot of us recognize of it not actually knowing what the problem is at all because it hadn't failed to fix it 5 times yet.
I believe their peak month was >13k issues opened, but worth bearing in mind: "This issue has been automatically locked since it was closed and has not had any activity for 7 days.".
I was running into something similar where I had it working on a single Python file. Maybe part of the issue is that I had memory turned off, and the conversation might have spilled into two conversations. I wasn't using version control either, but since it was a single file with about 1000 lines of code, it didn't seem too hard to eventually get all of my features back.
I'd be more interested to read on what 10% of the code is still getting written by humans and how it doesn't become a bottleneck at the pace they described.
Important/hard things are usually bottlenecks, its a good thing and usually by design. People should have more self imposed bottlenecks.
What are llm freaks in such a hurry for? Its like everyone is manically racing towards a finish line without realizing the finish line is you out of a job. Its like everyone is excited to be completely obsolete and poor.
My interest is mostly from the management perspective -- in 2025 I have seen a prime example when a heavily vibe-coding team chugged out half-baked products that other teams later had to deal with and this was not pretty and created all sorts of interpersonal issues; the same I fear may be happening here as well.
I'm looking at Claude Code and they chug releases daily and sometimes more often then this, if someone has to stop and code a critical path by hand or smth I'm wondering how this is handled.
--
On a separate topic, the "llm freaks" -- I know you intended this as an insult, but I'm actually proud to be called a freak again, and I thought this is the place where freaks are discussing things.
All this immense volume of changes in code but practically what does that translate to in the coding experience? CC isn't massively different from a few months ago, even vs the start of the year. I can list a few, like remote control which is nice but I wonder what all this amount of code churn actually achieves.
If so, it's phrased very unclearly: "merged pull requests to Claude Code's codebase more than doubled". They could be more specific that it's backend infra.
Here I am still unable to stop myself from doing line-by-line code review. I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) by doing it that I feel it would be careless not to.
My advice is to start a totally vibe coded side project. You will never gain confidence until you experiment, find the failure cases, learn to deal with them, etc.
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
I did this, but now I have a useful piece of software that I have zero clue regarding how it works. My only chance of "maintaining" it is giving it to another LLM to explain.
I am happy that I have a useful tool, but it simply isn't maintainable without a machine or me investing much time to understand it.
I don't see how the former is a sensible future for humanity. I am happy to say I won't be releasing this software I've created (I will find it useful myself), but deferring all understanding to a black box seems a poor future for brain activity.
I've been working on a large (~10MLOC) mostly vibe-coded project for months now. It's actually quite frustrating:
* The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself. Yes, I can ask it to look elsewhere for similar bugs, but even that's not super reliable.
* The user-facing text it generates is just as awkward as ever. I have to rewrite all of it, and sometimes it slips through. It's also hard to write text when you don't know what the code is actually doing.
* It doesn't have a good model of what kind of information is relevant to humans. It's just as likely to produce a paragraph of random jargon as a clear explanation of what's actually going on. It shows excessive detail for things that don't matter, and not enough detail for things that do. For example, it doesn't understand that $0 can logically mean "null" in terms of how humans think about things (e.g. "This item is free" vs. "This item is $0").
I kind of wish I hadn't gone down this path but now it's far too late, I can't really manually maintain this amount of slop. I'm quite sad if this is going to be the end state of this profession.
Ridiculous suggestion. If someone already takes issue with ai generated code (as well they should, it often is _still_ pretty bad), getting them to write more of it is useless.
Additionally, do you really believe that 'Now is the cheapest time it will ever be to experiment'? Why so? Because of token prices? Not a good omen if so.
I've done this, and always get subpar results. I cant get the LLM to make the minor tweaks I want to see, and now I have no mental model of the code to make the fix myself. Sometimes they arent minor things.
I have done this. It's exactly the reason I went back to doing code review. I don't look at the code with a fine toothed comb, but it does help me catch bad patterns early.
Is it responsible to move slowly when your competitors are moving quickly? Depends on how your competitors perform. If they perform well, then your "responsibility" suddenly becomes your extinction.
No one cares about good code. It’s a means to an end. Your code doesn’t need to look pretty or be well documented or any of that stuff anymore because people are not looking at it. The only thing people care about is performant and error free code.
Literally 50 years of engineering history and culture cares about whether code is good. There are libraries worth of books about it, not as a means to an app but for its own beauty. Just read “The Elements of Programming Style”, for example.
Do what you want, write apps with AI and don’t review code, it doesn’t impact me. The claim that no one cares about good code is demonstrably false, and that’s all I’m objecting to.
Code that isn't duplicated in 15 places, that doesn't read the whole database to fetch just 1 record, that doesn't do 15 database reads where one would suffice, that uses an optimized algorithm instead of an exponential one recreated badly from scratch... all that is also good code, and is just as important in LLM-slop era.
The way you phrase it makes me realize its almost like a GPL contamination. Eg, once you pull in GPL code, your project is infected and can never use a BSD (or other GPL incompatible) license.
Similarly, once you pull in terrible AI slop code, the project becomes unmaintainable by humans and you need to maintain it with AI.
No code is error free. All code will need a fix of some kind eventually. When it does, it's better that the internals are not a total mess. In the same way that you don't want the insides of your car to be total alien nonsense either.
if you take yourself out of the loop it will hurt your skills and career. Engineering blogs used to have some best practices you emulate, but the big ai companies you cannot mimic without their infrastructure
I'm somewhat surprised they would say Claude is writing 90% of their code at this point. Did they get tired of reporting new milestones, or has there been a plateau?
Claude Code is essentially the pinnacle of over engineering. Agentic harnesses are simply just not nearly as complicated as Anthropic seems dedicated to making them.
I'm really happy with Pi agent harness and last time I checked I could do everything CC can with my set of Chinese flash model coded Pi extensions.
It looks like they are specifically optimizing for incidents/reverts and shipping faster
> The goal hasn't changed: less human time per change, and no more reverts than before.
Is this Goodhart's law in action? Less reverts and less incidents was measured and became the target. Is this then still a good measure? I guess Revenue is a more direct measure, but we don't have access to that one.
From my perspective as a power user, the product has improved substantially over the past six months. It has gained new features that are actually useful, lots of UI/UX/perf improvements, and bug fixes as well.
New features have actually been remarkable. For me they are almost always in one of two categories: things I've always wanted (but never actually bothered communicating to the devs), and things I didn't know I needed but did. The product team is clearly very tuned in.
what are people using for automated code review these days? the claude code review product originally mentioned it cost "$15-25 per review" (which is absurd), but it seems like they've removed that from their docs with the launch yesterday. we've looked at various products in the space (coderabbit, greptile, etc.) but have gripes with each.
honestly i'm thinking it might be easier (and cheaper) to launch our own agent from GH webhooks and trigger `/code-review`..
We've had great success with a custom gh action on a self hosted runner. It sets up codex and all the same stuff a dev has on their machine:
So dbs, seed files, etc, and then just a review prompt and some skills and a few scripts.
It triggers on a new published PR or if tagged @bot-name. It's way better than the other services we've tried (which was admittedly ages ago) because you can get it to work exactly how you want it and it's dead simple.
With Codex, for the time being at least, you can use a subscription, so it's basically free if you're already paying for one. But on API pricing it's still not too bad as it doesn't produce that much output.
I set this all up in about half a day. Codex does 99% of the heavy lifting.
this: gh actions using an opencode harness (so we can use cheap models through OpenCode Zen). With the right model, and the right, scoped access, it's just a prompt away.
bugbot was costing us $1k per month for a small team. EVEN IF we decide to go with a paid or say something like GPT Sol for our custom harness it'll cost us maybe $100. We've already implemented all of bugbot's interesting features like cross-PR memory.
the cool thing is we can leverage this for things that bugbot cannot do. Just today I added a security-reviewer action reusing almost all of the code that we wrote for the code reviewer.
Depends on model, promt and amount of agents that review. I found that default code-review in Claude code is pretty mid for finding all issues especially if you downgrade model and effort.
During my evals on my own 3 agents that review in parallel they are much more cost efficient and cover 75% issues. Even on model like haiku it’s better score than default.
Multiple passes or raising effort will cover 90+. And this was tested on large diff, about 3500 LoC changes
in a vacuum, sure. the average pull request will have multiple commits once opened (i.e. to address feedback or perhaps it was opened too early), each triggering a review. it's very easy to rack up $100+ per pr (and an eng may open a dozen prs in a single day)
I've set up a claude code on a vm with a bunch of skills that can use git and the Azure Dev Ops MCP via a service user, the regular build agent pops a record in a db that serves as a queue/review and a bash script picks things up and fires off claude code in headless mode.
It's pretty effective and lets you do nice stuff like compare the code to the tickets, reply to comments and stuff like that.
Using a different lab's model to review the code to the one that wrote it. Claude model plans/codes, OpenAI model reviews, or vice versa, or DeepSeek or GLM or whatever.
I do it this way too. I think having the same lab's model review the code in a fresh session also helps, so I use both: I generally start a new Claude Code and a Codex and have them review the code with a focus on 3 things: 1. bugs, 2. cut corners, 3. code style.
This generally works for me. I think a solid spec is the key for the models to review against. I ask them to follow the spec, and if the spec doesn't say, follow first principles.
Also, if there is a user experience change, they need to notify me.
What I’m more worried about is feature identification and how to monitor them. How can you get the product to the next level?
Honestly I’m not worried about code anymore in my codebase I’ve not had bug problems for a while and incidents get fixed automatically.
I monitor new features with an agent who checks recordings.
Most actually good features need still 20-30 iterations to design the userflow to actually be interactive, but then there is also another challenge to translate usage patterns into features and how to present functionality with the least amount of friction as possible.
How to efficiently run monitors without wasting your token spends. A best 'monitor' best practice would be nice.
Still this article was very helpful and gave me a lot of new ideas.
The UI has improved substantially over the past six months. New capabilities are prominently featured and less-used ones are de-emphasized or removed. Every product goes through this type of evolution. The difference with Claude Code is that at the speed they iterate, it happens a lot faster and is therefore more noticeable.
It makes sense for Anthropic to pitch "more code = more better" but as anyone who's been in this game long enough knows, "less is more" often enough. And good luck coercing an AI coding agent to solve a problem by removing things.
Personally, I find Claude's degenerate urge to just spew uncontrollable amounts of code quite counterproductive. Yes I get it, you get paid by the token. I'd like to see more reasoning, more understanding, and less generating.
Truth is, we are still very very far from actual AI software engineering. Some people are working on useful tooling, and (predictably, like anything good and useful) it is taking a long time. LLMs are great, and hopefully they will help us accelerate building the tools that bring this profession to the next level.
30k PRs in a month (or whatever) is not the brag they make it out to be.
I think this is the future. Im more bullish than bearish. The only issue is that eventually human verification can no longer be possible and we will have to live under AI-verification assumptions....
They are in for a surprise when they become publicly traded and have auditors requiring actual humans signing off on any code that touches financial data.
This isn’t just billing related code. If they tell investors they have X active users, for example, the code that tracks active users is subject to the same standard.
I am happy people are testing these things out. I am also just going to wait a bit longer before really adopting things like this, I don't really trust it yet. A year ago this looked very different, and next year my guess is things will look different as well.
> Code review does two jobs:
1. Find and fix issues before they ship
2. Build shared context and codebase understanding across the team
So code review has become newspeak for “testing” and “education”?
Well not really testing. We the users are the testers and it falls to lucky us to really discover and find the errors, edge cases and security issues. The consequence of the rush to continually release…
Is the AI advance in the field of software testing nothing more than “now an AI agent can do a code review?”.
I built this PR Manager to help me review the PRs using Coding Agents. It sits on menu bar on macOS and has various lifecycle actions of PRs.
feedbacks are welcome.
There was a post some months back called "Lines of Code Got a Better Publicist" or something along those lines. This is just more of that. I am not impressed by 30 thousand "changes" per day in the slightest without significant added context. When I logged in and prompted Claude today, do I feel like I got +600,000 change value from when I logged in a month ago?
This might be one of those things were everyone on the team takes a 30 minute lunch instead of 60 and just nobody clocks out to get the 1 hour extra over time.. And everyone just kinda ";) ;) ;) Working from our desk while eating lunch ;) ;) ;)"
Absolutely, at this rate most software would be complete in a few hours. I may lack imagination but wtf are you changing? Endlessly refactoring? That’s spinning your wheels renamed as productivity.
lol right? So youre telling me they shipping 150,000 features this week? Or, maybe, they whack-a-moled 149,995 perf or bug regressions and shipping 5 new things. :shrug:
Right, but why do things this fast to begin with, especially for a product that's already publicly available?
I don't want to use a product that shifts under me as I use it several times a day. I want a product that actually moves slowly and works very reliably.
As an end user: why would I care about how many hundreds of releases a month are being pushed if maybe 1% of them have features I care about?
Genuine question I’ve been wanting to ask folks working on different types of things: how are your teams doing agent-assisted code reviews? Not automated reviews, but what techniques and tools are humans using to keep up with the larger volume of PRs that seem to trend larger and larger?
I think a lot of people like myself still want to feel in the loop but I don’t have the right tools to stay on top of the (generally good, productive, and still thoughtful) contributions from peers anymore.
Using agents for a first pass is a good way to speed up the review cycle. This way contributors can get near immediate feedback before a human needs to weigh in. This doesn’t replace the human review, but augments it. Each time your agent misses something, update your code review guidelines.
If their only metrics are revert/incident rate then they're missing the long-term, cumulative impact of compounding complexity due to poor architectural decisions.
Anthropic is doing some fascinating work. However, I find their communication style frustrating. It’s clear to me their employees have near unlimited token budgets. When Anthropic team members share their internal processes, I feel they don’t appreciate the constraints that many of us face in our work with budgets.
I don't understand how their work can be considered fascinating if an insight into their internal processes shows that they are completely detached from reality. This would mean that their work output is also detached from reality. "Fascinating" isn't the word for this.
I use Claude Code every day. And I find the product to work smoothly, and effectively. From the user's perspective, that's good software. I'm impressed with what they've built. Truly.
They are using a novel approach to write good software. That's fascinating.
TBH this community has lost touch with reality. AI is making people lose their minds.
Its unclear to me how at least Anthropic hasn't been doing what amounts to recursive self improvement for a while now.
They heavily use Claude for both code generation and code review of Claude's own code and related tooling. Is that not functionally Claude running recursive loops to improve the next version of Claude?
That's not news though. In fact almost no company beyond maybe 10 people that has a large central repository has no other smaller repos with other smaller production projects. They still always call it monorepo, because it approximately still behaves like one.
This workflow may not be accessible to token poor teams. Though I’m not all the way through the post, so I don’t know if token scarcity is even acknowledged.
I built a library to have a way to check Claude Code's work against API docs, and asked it to write many tests on top, but even then I found Claude Code would paraphrase and drift from the docs. I also found that it passed its own smoke tests but failed when testing against the real APIs, so reviewing Claude Code's code is tricky.
I used to do it but now I only review code organization and structure.
Mostly I define the product behavior I want and discuss the technical implementation beforehand.
Once that's in a spec I let it build in my durable implementation engine. Automated verification, review rounds from different agent types across different models, issues triaged into meaningful clusters and flagged for false positives before finally being fixed.
That loops until the code is up to spec.
There is no special secret, LLMs need repetition and multiple passes.
They'll one shot bigger and bigger things in their default harnesses but they'll be filled with issues no matter what.
I'm astounded that I kinda ended up building the same agentic workflow they are talking about in this article and distributed it amongst the org (eng and product/designers).
On the other hand, I would have expected Anthropic to have a drastically more elegant SDLC.
30k commits per day on a console app?? Are these all one-line changes?
I've written a harness myself and fair enough, it's more complicated than you might expect.. but surely at this point the claude CLI is pretty stable, and this must mainly be a ridiculous amount of test churn.
Or maybe their team of clankers is just creating busy-work to make the humans think they're essential. That's not unheard of in the human behaviours that they're trained on.
It's pretty bad, isn't it? I can't understand why that would need so many changes. It's like people/agents frantically working to look productive without any thought on the output or overall architecture. This is not something that should be copied or admired.
The furious pace of Ux “optimisation” that we see across the board (and especially at AI companies) is a hidden anti pattern. The time and frustration of learning a new ux is an acceptable burden when it frames real new capabilities or workflows. It’s not good to rework your ux user experience every week just because you can.
Not only that, but AI is categorically terrible at ux review. I don’t mean aesthetics, I mean attention flow, workflow in general and what the user anticipates as a logical path to feature discovery and exploration. Increasingly, features are hidden behind workflows that meet a single use case but fail to be universally logical or in many cases, even accessible at all outside of the test case.
So, people… use AI in code review. It’s actually pretty great at it if you do it right, and you can ship better software with fewer bugs than ever before. But please, don’t pressure your ux guy to “10x” with AI. In fact, now that you are “10x”ing your development cycle, you are probably going to need to hire -more- ux developers if you don’t want your product to feel like slop even if it’s gold under the hood.
I am not sure what it is you are saying. What I am saying: I would have supposed that Claude writes 99.x% of Claude's Code code, and that ~no code at Anthropic is written by humans anymore.
No. 90% sounds like a low number. That implies something else is writing 10% of the code? People? Other models? What on earth is writing that other 10%?
We have to test many different api's and models as we have over 600+ in our harness now, Sonnet 5.5 via cloud code provides the highest quality results.
Over the last few years we have added mcp tools, memory graph, md files and skills, automated everything to the point that on desktop the system runs itself, besides proceed/approved/explain.
We have 100's of PRs/jobs/projects and Claude builds down the list automated and only asks a few questions point form if needed.
This is like one of Google’s infrastructure blog. Google scale things are only relevant and appropriate for them. We had lots of bitter lessons from smaller engineering teams following the process with the justification that “this is how Google does it”.
Similarly, Anthropic can afford to do this because they have virtually unlimited tokens and labs to train models for their specific usecase. The rest of us need code comprehension lest we become dependent on continuous supply of tokens (which is a state these labs badly want to achieve)
So, unless you work at a lab, or have unlimited tokens guaranteed through the life of your company, it would do both the company and you good to continue to read the code and not let these 30k PRs per engineer be the norm.
An excerpt from the article makes my point
> Nobody on the team keeps a complete mental model of Claude Code in their head, and we don't think anyone needs to. When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today.
Even then do we really have evidence that what they are doing is reasonable or practical for them at all? They are edging on like 3 9s uptime, what leaked of Claude code looked like soup, tons of security issues, they can't build a sandbox to save their company, and last I heard they aren't even profitable without adjusting their numbers for arbitrary things.
Think I'm good on taking advice from companies that aren't making money.
Exactly. What they’re doing, repeatedly, is setting a narrative that this is how software engineering should be. Always be reliant on tokens. Don’t worry about uptime or errors - agents can address it, don’t worry about security - agents can address it, don’t worry about code comprehension - agents will always be there for you.
3 replies →
If they're cash-flow positive with 75% of compute going to research and training, what do you think will happen to cash flow when they shift to 50-50 training-inference?
Because "pacing the frontier" is more about flipping on the money printer switch than it is about safety.
> they ask Claude to write an explainer for exactly that slice
If their Claude is the same as our Claude, then this is a joke. Claude’s writing style is insidiously mind numbing. It takes constant pushing to get it to explain things in digestible English.
It also mixes important with unimportant information. It tends to think that a commit/bugfix is important and will happily list it together with important properties of the feature.
At least Opus 5.5 does that. Its writing is better than that if other models, though. I had other models explain stuff in an undigestable language, technically correct, but practically not what anything is about.
3 replies →
It's like it's trying to impress you with poetry in comment: "What the function does, it does well, and only just once"
Try having it explain something in “ISO Plain Language standard”. It made an enormous difference for me yesterday. The language will change from dense large paragraphs to simply-worded 2-3 sentence paragraphs. I was using gpt though.
Even getting it to write less obtuse English isn't enough IMO. It seems to have real issues with structuring documentation in a way that makes it easy to follow. Also it's a real demon for leaking its context as implicit assumptions into documentation.
v5.5 eliminated much of the impenetrable prose of earlier versions.
3 replies →
The people who work at Claude seem to talk in claudlish, so I doubt they care.
They pretty much fixed that. Plus you can influence the writing style to some extent. I tell it to use full sentences and avoid assuming I already understand the code.
Another thing is that it's very good at creating diagrams (I think someone wrote a skill for it so maybe not by default). I usually find the diagrams are even more valuable than the text.
Honestly I was skeptical but you really can just point Claude at some code and ask it to make a markdown + SVG explainer and it will do a shockingly good job.
I'd still rather read human documentation, but even before AI almost nobody writes any - especially for internal code. And they certainly don't make great diagrams.
2 replies →
If we as engineers didn't use similar terminology to describe any code not written by ourselves for the last few decades perhaps people would take it seriously.
2 replies →
[dead]
But the words are all right /s
If it was an exception only… but it is so common that I had to put ASD-STE100 into instructions to have precise answers.
I have heard from someone who works at Anthropic that they will have postmortems that sometimes devolve into "my claude says X" vs "well my claude says Y". Maybe this is the future of software engineering, but it seems pretty fucking sad.
Seems like a good time to get a third Claude to hear the cases of the first two Claudes and break the tie.
1 reply →
This is basically just being a manager in the before times. "My engineer said X!" "Well, mine said Y!"
Which is fine, whatever, I guess this is how I pay the mortgage off instead of having any clue what the hell Claude is actually writing. Hopefully I can semi-retire soon and go study dunnarts or take up welding or something else that's more rewarding than this nonsense.
I would love to ask them why this scale is even necessary for a conceptually simple CLI app like Claude Code, but they wouldn't be able to answer because they don't even understand the scope of it apparently.
They do have quite a few products now, the desktop apps, websites, cowork, cloud session sandboxes, API, enterprise, etc. have their own gotchas, but after a core development, the 30000 commits a day does not feel justified, or is that an online training dataset prep, that's hard to tell. That's why I feel with their endless tokens, it feels just petty marketing and that's sad because it does not bring much to our engineering table.
It's absolutely insane that they can't keep track of a CLI application that makes shell calls. This is so far from rocket science even mentioning the term is misleading.
9 replies →
It needs to gather a bunch of inputs and feed into the remote GPUs correctly and as efficiently as possible. Lots of room to optimize under the constraints that users love cheap tokens, demand a fair billing structure, and want responsiveness and low latency.
I don’t think you can infer the scale for Claude Code alone. They said it doubled but not what the starting point was. The 30k figure is for the entire Anthropic monorepo.
3 replies →
Is Claude code just the cli or does it include a bunch or backend services supporting?
2 replies →
> Nobody on the team keeps a complete mental model of Claude Code in their head.
For sufficiently large codebases in large organization that is how it has been with humans, who come and go. In many respects, What Anthropic is doing is a massive upgrade from the status quo.
Yes this is absolutely true.
Another challenge is getting developers to stop assuming they know what the code does without reading it! Having AI “fact check” various documents against the source will help immensely.
1 reply →
Really? Hundreds of billions of dollars in debt is an improvement? The product that this code is supporting can't even pay for its debt.
Jesus. Give me 1 billion dollars and I'll keep your entire codebase in my head + personal notes for you until I die. Hell, I'll even force my descendants to do the same. Now _that's_ an improvement.
3 replies →
It would benefit many people (especially Anthropic employees) if, after an LLM gives you a response like this, to go into a different session and ask another LLM if the explainer is correct. Not just so that you hopefully get the correct information, but also to see how easily fooled you can be by a plausible-looking, confident response about some code that you have no reference for. Use 5.5 on any thinking level, you will encounter this. The spaghetti mess that is CC likely makes this more of an issue.
Google's infra genuinely pushed the industry forward though, things like kubernetes and SRE methodology are generally useful even if you aren't Google. Reading this I'm not even sure this sounds like the best way to run an organisation with infinite free LLM tokens let alone an actual engineering org where it just an impact on development cost. The people who are getting maximum leverage from LLM reasoning and routing queries intelligently who've made big strides in reducing inference while increasing throughout are the ones who impress me the most.
[flagged]
I saw someone say the other day (in response to another blog post being well written) that Anthropic employees have a rule against using AI for writing/external comms? That seems clearly not the case here, normal people don't write like this:
> Most of the path is automated; a person's attention goes where the risk is
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
100%, although if people are getting 90% of their interaction from a robot, one may start to pick up some vocal tics and colloquialisms from machine world. a kind of robo-pidgin.
English isn't my first language, and I'm treating every opportunity to use the language as a way to learn, including extensive discussions with AI at everyday work. Sadly, I'm not noticing any unnatural wording in its replies.
I'm starting to worry about being gradually affected by AI's style through heavy use. It will make me so sad if one day my written piece sent to others is considered to be AI-generated, while it's written with real effort trying to make it sound natural.
1 reply →
Irony being, robots are averaging our best speech patterns. So we're really just syncing humanity's speech patterns, with LLMs acting as proxy.
2 replies →
I've definitely picked up some weird claudeisms. I even know the em-dash keyboard shortcut now.
2 replies →
Yuuuup. Which I guess makes it even harder to detect what's AI and what's not.
1 reply →
Pangram reports 80% of the text is AI-generated: https://www.pangram.com/history/72309aad-6e3f-4002-bdc5-c062...
I wouldn’t trust the numbers it gives https://news.ycombinator.com/item?id=49023243
1 reply →
[dead]
> That seems clearly not the case here, normal people don't write like this:
I think when you repeatedly expose and immerse yourself to such a degree in reading and writing language, it brushes off on oneself. So I could also absolutely believe someone from Anthropic wrote this.
agree 100 percent, I write prompts, and at some point I'm going to adjust my style when I see my prompt is ambiguous or something and I bet subconciously I'm going to try to talk like my robot pal when making these adjustments so I could see it happening
this definitely seemed like something claude generated, i know because i frequently do all my research in chat and then ask it to generate a doc based on what we talked about and when i read what it generates the verbosity, the style it writes in makes me delete the whole thing. I would never subject readers to that.
It felt weird reading it because every once in awhile there were some well written sentences and coherent paragraphs, then it fell back into Claudisms.
Another Claude-ism example from: https://www.anthropic.com/research/ecdsa-curve-origins
> This is not a cryptographic break, new attack, or weakness in the ECDSA curves.
I don't think that's a claude-ism because it makes sense to cover this point at the start of the article as a defence against overexcited journalists.
It's only a claudeism when it says "X is not a Y" when no reasonable person would have been expecting that X could be a Y.
1 reply →
Heavy emphasis on "every" like this is my new Claude smell, after honest and genuine seems to be reduced.
As a non-native, I see how susceptible I am to absorbing such style. There's no way at all for me to perceive anything wrong with the examples you present
It's a bit more subtle than things like "it's not X, it's Y". And even that isn't "wrong", it's just the way AI talks, and therefore an indicator that the author offloaded their thinking to an agent. I've read so much Claude output that it's easy for me to recognize its voice, the same way I could recognize a friend's voice in writing. The examples I quoted are a weird kind of circuitous sentence structure with a certain shape and rhythm that scream Claude to me. Here is how I might rewrite them in human speak:
> Most of the path is automated; a person's attention goes where the risk is
--> We automate as much as possible, focusing human attention on risky parts
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
--> We run reviewer evals against every incident
But that's just my interpretation and my voice. I can't know for sure what the authors would have said if they took the time to write it themselves.
1 reply →
Personally, a year or two ago writing like that did not really come across as particular bad or AI generated to me, although it was uncommon for humans to write phrases like that. Writing in an unusual way is not inherently bad, it can make your text interesting... but it is less direct and clear than alternatives. But the style becomes unpleasant if you keep seeing the same patterns over and over such as "X'ed against all of it"
It feels like it's trying to sell you something in every sentence.
3 replies →
The one that twigged me was in one of the first diagrams:
> Gradual release > The latest channel first; the stable channel trails it
Such an awkward, Claude-ish way of saying it
If you spend all day reading Claude-written "explainers" of your own codebase, I can imagine the "house style" would rub off on you.
What came first, "Claudish" or those who trained it? I wonder if Anthropic's employees originally wrote like that. It's definitely irritating to read, and IMHO more so than the ESL-grade English which was quite common before AI.
Claudeslop didn’t come from humans, it’s an artifact of running RL on slop outputs. If you keep feeding the slop output back into the model and then training on it, it amplifies its own unique weird vocabulary and behaviours.
The word “gate” was used.
load bearing comment
To be fair, that phrase does not appear in the article.
No wonder most people at Anthropic believe our jobs are in jeopardy, they've already made a system wherein the machines are treating them like pets.
I really don't get the tendency here from Anthropic to describe what's most likely churn? A high pressure amount of changes/PRs a day on a TUI to wrap a while loop? I'm not going to be a reductionist here so I'm going to cut it off there, I'm positive that the complexity of handling multiple terminal emulators is something that needs some handling (even if doing their own rendering engine was a self made problem). Yes I've vibe coded my own harness from scratch without writing any of it, I still know how it works.
I'm also not sure that they addressed point 2 in any actual capacity? They say "the system verifies itself" which is cool and is something that automated testing has done for decades, but they eschew the mental model of how the system works as a whole and prefer instead to have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
N.B. Changed "1000 PRs" to "A high pressure amount" to be vague since the article is vague here on what CC undergoes.
> have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
This is the crux. Take pause, because hidden here is a sea-change in our fundamental trust model of software...
There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug. By design, the system is too large for that, and everything hinges on Claude itself being able to find and understand the right slice of the system in any situation. As long as that very new invariant holds, everything will be all right. If it fails catastrophically, there is no recovery. You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
The new trust model is: We trust Claude to always understand the system enough to debug anything or make any reasonable change we could want. The risk might be worth the reward. But it's worth explicitly laying out the deal we're making.
Tbf, it's worth pointing out that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
"Tbf, it's worth pointing that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
The real difference as I see it is that in the human maintained old style system, the human would take things slowly and ensure they understand any changes. Claude let loose will just say "yes, I know how to do this" and bork the system, because it doesn't have a concept of humbleness in the face of complexity!
31 replies →
> You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
I love this phrase, and I think it captures the difference between the "60 years of legacy code" mentioned elsewhere here, and the vibe-coded monstrosity that we all know we're going to be encountering in the next 5 years. The fact that there were human brains with intent making all the little (short-sighted, myopic, ignorant, etc) changes in the legacy code means that you DO have at least some shot of gaining enough context to make safer changes.
There is no real intent in LLMs and particularly in blind vibe coding, so you'll end up reading stuff that works, or appears to work, but there was no human brain behind the change for you to model and thus predict.
6 replies →
most large software companies are already too large for anyone to have a complete mental model of.
The idea that someone at say, Google, will be able to fit even half of the system architecture in their heads is not realistic. It also hinges on being able to find and understand the right slice of the system in any situation.
But we have all sorts of guardrails to make that easy - i.e logs, traces, etc
14 replies →
> There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug.
At this point, why even bother with source code? Source code is designed for humans to comprehend. If humans are no longer the intended audience, why not have machines output and maintain some more efficient intermediate representation of computer programs?
I fixed many bugs over the last years with help of claude code in an unfamiliar codebase. it is never just Claude alone so we don’t rely on AI understanding 100%. It is whatever AI can understand plus what I do. If you use AI right your understanding will also grow over time.
> But.... That's not knowledge?
Thank you! This is a huge sticking point for me. There is no knowing the code without experiencing the code. If nothing else, you have to read the tests to know they verify what you think they do. Even then, you might know it does what you want, but can’t know it doesn’t do anything it shouldn’t.
There are no shortcuts to true understanding.
> wherein the machines are treating them like pets
I know some people will groan hearing the phrase again, but: "Reverse centaur." A centaur is a synergistic combination of two things, but now those things are in the wrong position: The human is the computer's tool.
Naturally, how "wrong" it is tends to depend on whether you ask the people who do all the work or the people who own all the computer-capital.
When you get to the point that you are doing 1000 PRs I have to seriously wonder what even are you building, I hear about this a lot from a number of companies, but their “too many commits / too many PRs” problem never seems to give anyone from the outside the feeling they are innovating at groundbreaking speeds.
My favourite quote at the moment: "May not man himself become a sort of parasite upon the machines? An affectionate machine-tickling aphid?" Samuel Butler, Erewhon, 1872
I think it's just a different focus on what matters.
There's clearly enormous (human) work that goes in to building and tuning their models.
The software they develop around their model is, relatively, trivial. It's not clear that enormous amounts of (human) energy need to go in to that code.
It seems largely a function of applying human energy towards the things that are marginally most important to generating you money and putting you in front of competitors.
>The software they develop around their model is, relatively, trivial
30,000 changes a day is not my idea of trivial.
6 replies →
I mean, sure we can assume that (although the Anthropic article from the other day say that claude mostly builds Claude) but this is the Claude Code team which, I again am assuming so I could be wrong, focuses on the TUI/CLI interface to those models, not necessarily the models themselves. That coupled with the fact that this is how a large portion of their clientele interact with their moat, it would be odd to me if they didn't apply significant work towards this thing to keep the hype or productivity train moving
> I really don't get the tendency here from Anthropic to describe what's most likely churn? 1000 "PRs" a day on a TUI to wrap a while loop?
From the article, the 30,000 changes a month are in the monorepo, and Claude Code is in a separate repo. So it’s not the TUI that has 1,000 PRs a day.
I could easily see 1,000 PRs a day on the backend. Big tech was doing more than that pre-AI.
When I worked at Facebook (pre LLM) we were around 50-100k per day. I imagine it's much higher now (quantity, probably not quality).
Fair enough, I did invent the 1000 a day because the article was vague, I'll edit my submission accordingly to also be vague
The Anthropic post the other day mentioned part of what helps Claude Code unlock some of this is the sheer ability to use more tokens than the rest to get something going and efficient.
> No wonder most people at Anthropic believe our jobs are in jeopardy, they've already made a system wherein the machines are treating them like pets.
I guess it wouldn't be surprising for people prone to AI psychosis to be attracted to working at the frontier labs, resulting companies that are heavily in the throes of mass AI psychosis.
Being tongue-in-cheek here.
This is a good answer
[dead]
The ownership component is the toughest and least enjoyable part of coding with ai. AI saves the time of coding, but 100% of the responsibility remains on you. You then have to invest an imbalanced amount of time to understand and fix the issue vs the time it took to generate the code. Then on top of that is the emotional feeling of something you “owned” messed up, but it wasn’t really your doing - it was the AI that did the keyboard work and now I need to parent/manage the AI. This destroys trust in AI and the cycle restarts. But we keep coming back to the slot machine as we often get those returns where things just work, but like gambling, you’re not satisfied by those wins and you’re immediately on to looking for the next fleeting high.
I think the question is then "how do I gain more confidence this code is correct?" This doesn't have to be reading through everything line by line, it can be writing more tests, or working with the LLM to dig into how certain edge cases are handled etc. There are a lot of things you can do to improve your confidence in a piece of code.
> You then have to invest an imbalanced amount of time to understand and fix the issue vs the time it took to generate the code.
Obviously no amount of training will make AI human but: how is this different from you being the leader of a team whose output you own and are accountable for?
The difference is that now people who signed up and were hired as engineers now have to do what managers used to do. It's also a different pay grade when managers are accountable for a team vs and engineer for Claude instances.
In a weird way, it reminds me of playing chess. If I play a human opponent I can be thoughtful and deliberate. If I switch to playing a bot, I respond to the speed of the responses no matter the level and get crushed.
It takes a while to settle in and play deliberately. I also need to switch gears because bots play much differenly than human opponents - they don't fall for bait, they don't have time pressure at all, they don't leave pieces uncovered by accident.
I think of coding with AI is similar, you need to have a framework where you build understanding along the way from plan to review. It is untenable to roll out a 6000 line feature then try to review it, even if the LLM is happy to do things that way, mere mortals can't keep up. But if you stay in your lane and code solidly, AI can be a straight speed boost. I feel like most people despairing over handing the reins to AI simply haven't found their personal sweet spot.
And so the wildly unprofitable AI company explains how it token maxes it's code and review process.
Authors should really try and view this article from the perspective of someone who has to think about cost to use the models.
I look at it as a giant experiment. Any research tends to be very investment heavy, and people are given a lot more leeway to try things which are wildly unprofitable.
I'm glad they're doing this, because it lets me see an extreme of LLM development.
We spend a few thousand dollars a month in AI code review costs.
Compare that to the cost of human code review which can take up a third of an engineer’s time. You’re looking at $50-100K a year per engineer fully loaded in code review costs.
If AI assists in code review (not even replaces – we still do human review ourselves), you could spend thousands a month per engineer and still come out ahead.
Ironic that this comment talks about cost without weighing the numbers.
It’s gotten pretty good too. My employer has a team that wrote their own code review bot with the checks and values we care about and it has gotten quite good over the last six months.
[flagged]
[flagged]
"Anthropic told investors that it will profit on a measure called adjusted operating income.
"According to the report, adjusted operating income excludes stock-based compensation, which is not unusual for technology companies. However, the report also quotes its gross margins - above 80% - exclude revenue shared with distribution partners, as well as the cost of training its models."
From https://www.morningstar.com/news/marketwatch/2026091414/the-...
2 replies →
How so? Can you link to the numbers?
Antropic's revenue is up [1], [2], [3] and if you calculate adjusted operating income (by excluding most expenses like training costs) then by that measure it is profitable. However, if you factor in all the numbers it is operating at a loss.
[1] https://finance.yahoo.com/technology/ai/articles/anthropic-v...
[2] https://fortune.com/2026/09/29/anthropic-leaked-ipo-prospect...
[3] https://www.cnbc.com/2026/10/15/anthropics-third-quarter-rev...
Source? Only reports I’ve seen was they were profitable for one month in last quarter due to xAI discount of renting out colossus . Be interested to learn otherwise.
That's factually incorrect, and shouldn't be allowed here.
Profitable as in after expenses.
https://youtu.be/T-oXyXwD6sE?si=tauzXVyGmQ6MhI_5
2 replies →
We're talking about being actually profitable, not fake profitable.
“I think that, every time you see the word EBITDA, you should substitute the words "bullshit earnings.” ― Charles T. Munger
And what they're reporting is adjusted even further than EBITDA
Yes when you subtract expenses and capex
[dead]
Anthropic has amazing models. But they should really stop commenting on software practices. Most of what they say seem out of touch with what's happening in software shops. A lot of it seems hype driven. And the entire message looks to be written by AI.
> But they should really stop commenting on software practices.
I disagree. Many of us have an emotional attachment to the way we do things and have difficulty letting go. Anthropic is showing a different way of doing things. And I am not going to judge them too harshly if the final output is good. I use Claude Code every day. And it is good. Ahead of the pack.
You're failing to differentiate what Claude Code means here. The models, no one argues that they're not good. If you mean Claude code the local agent, there are now dozens of smaller, faster, lighter harnesses that do everything the Claude tui does and more.
Yes, this is a different way of doing things, and it's clearly working for them, but there is no direct line between that and a "ahead of the pack" product.
I don't understand saying you have an emotional attachment to doing something. Normally you do things in a certain way because it is functional and works. What we're seeing from these LLM companies is them proposing new methods of working which highlights insane ways of "working" (they don't "work") since they're reliant on an LLM, which is just self-marketing.
Given the internal models that they have access to, I'm more worried about software companies being out of touch with the next generation of models will enable. Anecdotally I've seen that it takes a while for an organization to adjust to new LLM capabilities, even if everyone had access.
It’s all marketing nonsense. Along with the “AI is dangerous” and “we’re all going to lose our jobs” messaging they do, it’s just to sell the services.
> Low - Docs, copy, internal-only features, additive changes behind a flag
> None required once automated review and security checks are green
How anybody can let an LLM write docs and not even read them is pure insanity.
It’s writing the docs for other agents obviously
From the article: "merge with at most one human in the loop, who can be the author". Curious to see how this handles enterprise compliance requirements and insider threats?
- How was this approved for SOC2 compliance? Our SOC2 reviewer explicitly mentioned that all code needs to reviewed by at least one other human before it goes to production. Claude Code says they are SOC2 compliant (https://code.claude.com/docs/en/security).
- This assumes all engineers will not do anything malicious. What is preventing a Claude engineer from spying on their ex's or some other companies Claude searches by pushing malicious code? Recently an OpenAI engineer was caught planting a recording device at their own office.
I’m sure it’s not compliant. We have the same requirement for soc2. However the audits are pretty light and a company big enough can afford to have an auditor who looks past those things.
i think SOC2 is about audibility more than anything - my guess - and i’m not an internal auditor or anything - is soc2 needs auditing for human changes, but a machine is just a “system behaving on its own”
No human can stay on par with 30k pull requests per day ... they simply don't have control on what's going on and rely on AI itself to do the job.
PS: I really want to know what is the churn rate (and the cost per LOC) here. My gut is saying the cost per LOC is insane, and they are basically rewriting over and over the same code.
30k PRs is such a meaningless figure though, you can't make anything of it. It entirely depends what's living in that repo. Adding localized strings, configs, things that change frequently, can easily move numbers like this by orders of magnitude. How big are the PRs? Maybe there's an auto-bug-fixer agent that runs around the codebase making thousands of single-line fixes per day. Who knows. The number means nothing in isolation. We used to scoff at bad managers for measuring the performance of engineers using lines of code, now we're using the same figures to judge AI. We've become the bad managers.
Actually LOC is a decent measurement, it becomes a bad measurement when you use it in the wrong way, like any other measurement.
Agree that 30k PR alone doesn't mean much, but there aren't other details on what's inside. Based on I saw on how LLM programming works, it's hard to believe that there are single-line fixes. If I have to bet I'd bet on 200 LOC on average, that is 6M/day. Even if only a small portion reach production, it's still insane.
3 replies →
> One colleague said slowing down does lower your error rate, but mostly because you're doing less of everything
That’s not how rates work
They aren’t claiming that the doing less by itself changes the rate in this sentence. They are saying that the rate changes when you slow down, probably due to having more time to focus, which in my opinion is likely.
Haha, yeah.. Also with most real world, vs LLM FOMO hype fantasy land, consumer software has two very real tiers of bugs; regressions, and bugs in new features.
Regressions are usually the absolute worst.
Up to now we've just been eating shit on the DX side as far as regressions in Codex and Claude Code because everything is all "go-go-go OMG I don't want to fall behind" and also.. Access to new models is tied to harness updates.
But a ton of industries have users that are not so willing to eat the regressions in the name of more features and changes than they can keep up with.
Edit: Also moving too fast.. You may have been less likely to "get it right" than ever before leading to design churn. Again, depending on industry and users, they may not be very tolerant to releasing the 5th complete overhaul of some feature in so many weeks..
How is the ability of a human to understand the code via hands-on familiarity not a direct dependency of their ability to do manual code review? I don't get it. Having claude write an explainer (something I do a lot!) doesn't get you to the point where you can confidently assess a complex diff. And you've funneled your priors on what the diff says and does through the same system that already reviewed and approved it. This article isn't bad but it doesn't give enough weight to the human context-building part of the problem.
I think there is no magic trick here. The answer is simply that humans will no longer understand the code.
Although, following Anthropic’s ethos and taking this to its conclusion, I’m not even sure why they need human reviewers at all. What, they bring in a human to look at a diff of an “important” section of code that was written by agents and has changed a dozen times since last week through 12 PRs of churn? Obviously they don’t know what’s going on. So what’s a human to do? Do what the other approvers (if any) did: fire up a Claude to command it “verify this change has no regression; check carefully for edge cases”, then come back after a half hour of High Thinking to see “Change is clean (with some non-blocking findings below, in order of significance)” and click Approve. Not sure what the purpose of the human is in such a process.
Fair assumption. I think humans, at least those who like to code and build solutions, will become better builders than coders. Understanding the code will become less important, but knowing how to put it all together will be essential.
Yes it used to be said, the code is not the asset, the people who know what the code does and how to change it are the asset.
Now we’re just going to outsource our capacity to design and review because it’s faster that way, as if the thing that was wrong with the software industry was that releases weren’t being pushed out often enough.
I agree with you. A code reviewer from the same company, even with different models or efforts, shares the same blind spots. To review my code, I use a different company; if I code with Claude (usually Sonnet 5.5-medium), Codex reviews it, and vice versa. I get much better code this way. But more importantly, I always tell the reviewer to build its own test-scenario ledger and refute the coder's claim that it is done. And the verdict is shared with the coder, but the coder cannot reject it.
[flagged]
At 30k commits a day, is there anything that can meaningfully be called review?
I'm thinking about high volume mass manufacturing as an example here. I like those "How it is made" type YouTube videos. You see a production line that has some kind of QA process where cameras + automated systems do spot checks at high speeds and these crazy actuators are punching out bad examples. These processes often also involve spot checks where some statistical sample is selected and viewed carefully.
I don't think we've gotten to that level of reliable review with code, but I don't see any reason we can't continue moving in that direction.
I really like those videos too, but a very big difference is, in manufacturing QA, the system outputs a very high number of copies of a very small number of possible types, so spot checks work fine from an operations research point of view, as the variance in output has a known upper bound.
For code, I think things go in the opposite direction: the system outputs artifacts that are all different among themselves (even more so now that we no longer distribute software via physical copies. Well, bits are always physical, but I mean we no longer ship floppies/CDs/DVDs).
That said, at least in my experience, code review has never been a direct part of factory-line-type QA. That QA is typically done by treating software as a black box and just verifying that it complies with a spec (often times based on a very loose definition of "spec" ...), and this is something that works equally well regardless of whether a human or a machine wrote the software. The type of QA that is done (among other things) by code reviews is checking whether the software's design is such that it will be economically feasible to maintain and extend it, for example.
The one part the code reviews takes a part of (but is in no way error-free) and that you just can't do with black-box testing (or it is much harder to do) is to look for things like backdoors. As our industry moves more towards "agents write software and tests validate that it works," I think a lot about "Reflections on Trusting Trust" and the new and wide attack surface added by people no longer reading the code shipped on their behalf.
That’s a category error. Software isn’t stamping out the same part again and again; if it were review would be simple. Analogies often mislead.
Of course Anthropic would love you to think you are heading towards that future so we have to bear in mind their livelihood depends on LLMs being the future of work and 30k changes a day being normal.
But why would we want 30k changes a day in a given product, what possible justification could there be?
4 replies →
Those manufacturing videos are of someone making the exact same thing over and over. Months and years of testing go into ensuring that there is no change in the system, and continuous monitoring is put in place so that if any changes slip in, it can be undone. The design is set in stone, as it should be, and variations are bugs. There are months to years of cheap prototypes, iteration, design changes, pipeline tuning, and verification to ensure that nothing will need to change for the next million items.
If you want a software analogy to those devices getting churned off the assembly lines, it's the TCP streams coming out of your network card, with crazy checksums punching out bad packets.
We're orders of magnitude better at reliability with code, but we're regressing on that metric rather quickly. And the comment above deeply misunderstands both manufacturing and software.
9 replies →
Those machines are not doing design work. They automate production of designs that had long development and testing lifecycles. Code is design. The compiler is the automated production process.
Honestly systems like this resemble real engineering more than manual human review does
I dont believe it, its FOMO marketing to me.
Why not? Code review is a crazy time sync. If I'm working on a project without code review (e.g. a personal tool) using Claude I can easily churn out 100 commits per day. And it isn't even slop these days.
It does make me wonder what on earth they are working on. Feels like most of their products could be basically finished already.
On the other hand Spotify somehow employs 3000 programmers, and that's just a music player.
It's conducting/steering/oversight/supervision, not reviewing.
> We don't have a formal process for this. Nobody on the team keeps a complete mental model of Claude Code in their head, and we don't think anyone needs to.
Nobody on the team keeps the mental model of Claude Code as it is today in their head, so nobody on the team is able to reason and decide about Claude Code in the future, is that right? So no long term architecture strategy can exist as no current architecture understanding exists. While it seemingly does work for a system which is a TUI, would the same be true for a complex distributed system today?
'Nobody has a complete mental model of <insert name of system>' has very much been the norm since the invention of the two week's notice. What's actually new is that an architecture strategy can now be cheaply reverse engineered out of any system, adjusted where needed and immediately applied to the development process in days or hours since 'the team' is now a single dude or dudette with a fleet of robots which can see the whole code base together with its durable arch docs at once.
It doesn't do all that much - so nobody being able to fit it in their head is not a good sign.
Before all came to this, I've been promoting reviews at $dayjob not as gatekeeping but as method to exchange information and build more team spirit. We are an internal service team working by "projects", which makes us a very disjointed team with barely any communication. (Part of it is also German work culture.)
And now this. Suffice to say, my efforts did not bear fruit. I feel more isolated than ever before.
Reviews were always meant to be about more than just finding bugs, thanks for mentioning that.
Nowadays I'll ask a question to, for example, try to understand why was a particular abstraction chosen, and I'll get a response from a robot, sometimes without even a disclaimer that the response came from a robot (it's still somewhat easy to spot due to the shape and style of the prose).
It does make me feel like there is nobody on the other side, especially working remotely, so I think I relate to your feeling of isolation.
Sometimes I care about how some change impacts the broader application moving forward, not really the specific technical tradeoffs for one construct or another, and that's, more often than not, very lacking.
Does anyone ever read what code these models produce? 50% or more is just crap: meaningless tests, convoluted code, a lot of indirection, unreadable with a lot of bugs. I am not surprised that they merge as much as they do, they just dont know what the code does but see it is somewhat working. Most people who are impressed by it, I am sure, are just shit coders themselves ngl
> Most people who are impressed by it, I am sure, are just shit coders themselves
The less you understand code, the more you're impressed by coding chatbots - in part because you're less likely to read the bot's code, and if you do, less likely to realise how poor it is. Code gets judged good simply because it exists and costed little to create.
It's funny to me that they use Claude Code as advertisement for their coding models. This software is mediocre at best, full of useless bloat and really only good enough for the use case (unless, the use case is advertising their models to vibe coders, then it's pretty good and I'm not the target audience)
Under low risk, no reviews required:
> additive changes behind a flag
That's low risk when the flag is off. Is there suddenly proper code review before the flag gets turned on? That's when the risk is actualised.
Also, when they say "risk" - risk of what? Bugs and security incidents? What about other risks: future rework, slowdown of development, obfuscation of bugs, abstractions that will end up being made leaky to work etc. Are these all not considered because they are confident Claude can manage all of that?
So many questions
I don't know if this is just the Claude Code team that works on the Agentic harness or the one that does the backend model hosting. But if it's the former, I think posts like this are exhausting because their stack is so simple that a setup like this is possible.
I would love to create agentic feedback loops like this but my project (and probably a lot in the software industry) are compiled. Compiling not only takes a lot of time, but we have to carry around nearly 200 GBs in toolchain dependencies, require access to license servers for 3rd party software used in the toolchain and to run the full integration test suite on the project can take hours even with parallel testing in the VM while using multiple VMs. And these VMs aren't small, 64 core/128GB. So outside of unit tests, we can't really merge code fast and test it with high confidence without spending an absolute absurd amount on VMs in order to enable the agents to move quickly.
This was an exhausting post. both the writing style and the thought that this is what bleeding edge currently looks like.
Some of the most capable engineers in the world high on their own supply… I have no doubt they think this is aspirational but even as I know LLMs will forever change how we code, I’m not entirely sure 30,000 PRs for a coding agent is a volume that makes any sense.
All good ideas, I have been doing something similar for half a year. The most important thing is to reduce bugs and ensure the code does what you wanted. Writing code tests, logging, interacting with the running app are essential. For intent realignment I like to have the agent re-read all user messages posted in that project. It only takes a few minutes even on a 10,000 message project. Then use this fresh derivation of user intent to check code and tests. For long spans of work I prefer to generate a todo list which remains as a document in the project. Each closed gate tracks feedback from the agent on what happened. This task file is also reviewed twice - once before implementation and once after. So the formula is - review agents - tests - user message logs.
i am tired of listening about other ppls agent setups. Everyone thinks they've discovered something cool and unique but They are all the same.
Its like telling other ppl about your dreams. Not interesting.
I used to keep a dream journal because I thought it might be interesting to recall all of my dreams.
Eventually I did, but it was hours and hours of effectively random stuff happening that I had to write down or dictate with speech to text.
It was a waste of time and I'm now glad I forget almost all of my dreams. My point is even my own dreams are uninteresting.
I'm not that interested in my own agent setup either - usually the tools become better on their own after a few months and render it all obsolete.
You don't need to read my message if you think you know what is the best formula for your AI usage. Whenever someone complains to be "tired" of reading some online message I take it as an act of verbal aggression. I rarely disclosed my formula online, and probably never will again after such reaction. Do you want to regulate HN? Was my message off?
1 reply →
Sir, have you heard of our lord and saviour, Markdown?
Much like the output of agents themselves
So true.
For real, my eyes completely glaze over, there is nothing interesting in it.
Mfw tui needs a billion prs per day.
Opus 5.5 is obviously stellar but come on what are you doing lol
They need to add tamagotchi pets to the TUI so you stay awake while approving commands.
Staff software engineers(because all the seniors and below have already been sent to work at the soylent green factories) will have AI monitoring software installed on their computers.
If it detects they aren't paying attention, fall asleep, or haven't approved enough PRs per hour; off the work at the factory.
Well, that explains the Github repo with 5000 issues, maybe about a third of which are random regressions of previous behavior.
I snorted laughing at this. "We are experiencing tremendous growth" and it's just a codebase with 1,000 agents endlessly adding markdown documentation for every 1 line change.
I just learned that manually invoked skills MUST be the _first_ thing in the prompt, e.g. you can't have `disable-model-invocation: true` and say "Hi, /grill-me on X".
Worse, everything after the /grill-me is injected a second time into the prompt as a <command-args> so your prompt is literally added to the context TWICE. (first time in the original user prompt.)
I swear Anthropic is intentionally bloating your context so you burn tokens on useless bullshit like this.
Oh /loop was the crowning achievement. Basically guarantees you'll run a prompt at max context without caching over and over.
I have done exactly this as recently as yesterday, so I’m not sure why it doesn’t work for you.
2 replies →
Codex had over 15k issues last I checked including legit months old issues with the sandboxing/bubblewrap config.
Human contributions to issues, comments or submissions, were drowning in a sea of AI vomit. Tons of the issues had those AI issue submissions with the classic confidence of having identified the problem, but that smell I know a lot of us recognize of it not actually knowing what the problem is at all because it hadn't failed to fix it 5 times yet.
https://i.imgur.com/lYjn8xq.png
their claude code changelog 'fixed' % over time.
https://i.imgur.com/KJcem6c.png
I believe their peak month was >13k issues opened, but worth bearing in mind: "This issue has been automatically locked since it was closed and has not had any activity for 7 days.".
I was running into something similar where I had it working on a single Python file. Maybe part of the issue is that I had memory turned off, and the conversation might have spilled into two conversations. I wasn't using version control either, but since it was a single file with about 1000 lines of code, it didn't seem too hard to eventually get all of my features back.
> Security gets its own automated review too. ... You either fix it or close the thread with a short reason, which is how you clear a false positive.
I hope this is only for low-risk false positives. Dismissing a critical finding as a false positive should still involve AppSec.
You could improve revert rates while simultaneously introducing more security vulnerabilities.
And the linked "claude-code-security-review" explicitly states in the README
> This action is not hardened against prompt injection attacks
That's a significant limitation for an AI security review control, no?
I'd be more interested to read on what 10% of the code is still getting written by humans and how it doesn't become a bottleneck at the pace they described.
Important/hard things are usually bottlenecks, its a good thing and usually by design. People should have more self imposed bottlenecks.
What are llm freaks in such a hurry for? Its like everyone is manically racing towards a finish line without realizing the finish line is you out of a job. Its like everyone is excited to be completely obsolete and poor.
My interest is mostly from the management perspective -- in 2025 I have seen a prime example when a heavily vibe-coding team chugged out half-baked products that other teams later had to deal with and this was not pretty and created all sorts of interpersonal issues; the same I fear may be happening here as well.
I'm looking at Claude Code and they chug releases daily and sometimes more often then this, if someone has to stop and code a critical path by hand or smth I'm wondering how this is handled.
--
On a separate topic, the "llm freaks" -- I know you intended this as an insult, but I'm actually proud to be called a freak again, and I thought this is the place where freaks are discussing things.
1 reply →
All this immense volume of changes in code but practically what does that translate to in the coding experience? CC isn't massively different from a few months ago, even vs the start of the year. I can list a few, like remote control which is nice but I wonder what all this amount of code churn actually achieves.
Claude Code is not in the monorepo. The monorepo is for their internal tooling and backend infrastructure.
If so, it's phrased very unclearly: "merged pull requests to Claude Code's codebase more than doubled". They could be more specific that it's backend infra.
1 reply →
Here I am still unable to stop myself from doing line-by-line code review. I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) by doing it that I feel it would be careless not to.
My advice is to start a totally vibe coded side project. You will never gain confidence until you experiment, find the failure cases, learn to deal with them, etc.
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
I did this, but now I have a useful piece of software that I have zero clue regarding how it works. My only chance of "maintaining" it is giving it to another LLM to explain.
I am happy that I have a useful tool, but it simply isn't maintainable without a machine or me investing much time to understand it.
I don't see how the former is a sensible future for humanity. I am happy to say I won't be releasing this software I've created (I will find it useful myself), but deferring all understanding to a black box seems a poor future for brain activity.
4 replies →
I've been working on a large (~10MLOC) mostly vibe-coded project for months now. It's actually quite frustrating:
* The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself. Yes, I can ask it to look elsewhere for similar bugs, but even that's not super reliable.
* The user-facing text it generates is just as awkward as ever. I have to rewrite all of it, and sometimes it slips through. It's also hard to write text when you don't know what the code is actually doing.
* It doesn't have a good model of what kind of information is relevant to humans. It's just as likely to produce a paragraph of random jargon as a clear explanation of what's actually going on. It shows excessive detail for things that don't matter, and not enough detail for things that do. For example, it doesn't understand that $0 can logically mean "null" in terms of how humans think about things (e.g. "This item is free" vs. "This item is $0").
I kind of wish I hadn't gone down this path but now it's far too late, I can't really manually maintain this amount of slop. I'm quite sad if this is going to be the end state of this profession.
1 reply →
Ridiculous suggestion. If someone already takes issue with ai generated code (as well they should, it often is _still_ pretty bad), getting them to write more of it is useless.
Additionally, do you really believe that 'Now is the cheapest time it will ever be to experiment'? Why so? Because of token prices? Not a good omen if so.
3 replies →
I've done this, and always get subpar results. I cant get the LLM to make the minor tweaks I want to see, and now I have no mental model of the code to make the fix myself. Sometimes they arent minor things.
I have done this. It's exactly the reason I went back to doing code review. I don't look at the code with a fine toothed comb, but it does help me catch bad patterns early.
3 replies →
[flagged]
25 replies →
This is why I like GitHub Copilot's autocompletion, I'm right there guiding and policing the code as it appears.
You're unable to stop being responsible? Tragic!
Is it responsible to move slowly when your competitors are moving quickly? Depends on how your competitors perform. If they perform well, then your "responsibility" suddenly becomes your extinction.
4 replies →
No one cares about good code. It’s a means to an end. Your code doesn’t need to look pretty or be well documented or any of that stuff anymore because people are not looking at it. The only thing people care about is performant and error free code.
People saying this are telling on themselves that they have no idea what quality code is. It's not about looking pretty or nice variable names.
Code quality matters to AI just as much as it did to people
13 replies →
Literally 50 years of engineering history and culture cares about whether code is good. There are libraries worth of books about it, not as a means to an app but for its own beauty. Just read “The Elements of Programming Style”, for example.
Do what you want, write apps with AI and don’t review code, it doesn’t impact me. The claim that no one cares about good code is demonstrably false, and that’s all I’m objecting to.
2 replies →
Good code is a means to the end of the code being error (and vulnerability) free, maintainable and performant.
And if people are not looking it, those people don't care about their customers.
2 replies →
Not true, LLM’s will struggle with badly organized and designed large codebases just the same, though likely with less errors and profanities.
1 reply →
> No one cares about good code. It’s a means to an end.
I do, my coworkers do, my friends do, etc etc.
3 replies →
Code that isn't duplicated in 15 places, that doesn't read the whole database to fetch just 1 record, that doesn't do 15 database reads where one would suffice, that uses an optimized algorithm instead of an exponential one recreated badly from scratch... all that is also good code, and is just as important in LLM-slop era.
The way you phrase it makes me realize its almost like a GPL contamination. Eg, once you pull in GPL code, your project is infected and can never use a BSD (or other GPL incompatible) license.
Similarly, once you pull in terrible AI slop code, the project becomes unmaintainable by humans and you need to maintain it with AI.
1 reply →
No code is error free. All code will need a fix of some kind eventually. When it does, it's better that the internals are not a total mess. In the same way that you don't want the insides of your car to be total alien nonsense either.
if you take yourself out of the loop it will hurt your skills and career. Engineering blogs used to have some best practices you emulate, but the big ai companies you cannot mimic without their infrastructure
I'm somewhat surprised they would say Claude is writing 90% of their code at this point. Did they get tired of reporting new milestones, or has there been a plateau?
[dead]
Claude Code is essentially the pinnacle of over engineering. Agentic harnesses are simply just not nearly as complicated as Anthropic seems dedicated to making them.
I'm really happy with Pi agent harness and last time I checked I could do everything CC can with my set of Chinese flash model coded Pi extensions.
Token peddler says best way to do something is using more tokens.
This is not HN-worthy
It looks like they are specifically optimizing for incidents/reverts and shipping faster
> The goal hasn't changed: less human time per change, and no more reverts than before.
Is this Goodhart's law in action? Less reverts and less incidents was measured and became the target. Is this then still a good measure? I guess Revenue is a more direct measure, but we don't have access to that one.
From my perspective as a power user, the product has improved substantially over the past six months. It has gained new features that are actually useful, lots of UI/UX/perf improvements, and bug fixes as well.
New features have actually been remarkable. For me they are almost always in one of two categories: things I've always wanted (but never actually bothered communicating to the devs), and things I didn't know I needed but did. The product team is clearly very tuned in.
All in all, I'd say their approach is working.
I'm curious what has improved so much. To me Opus 5.5 is very good but so was 4.6. I try to forget the versions in between...
not even one tangible example?
3 replies →
what are people using for automated code review these days? the claude code review product originally mentioned it cost "$15-25 per review" (which is absurd), but it seems like they've removed that from their docs with the launch yesterday. we've looked at various products in the space (coderabbit, greptile, etc.) but have gripes with each.
honestly i'm thinking it might be easier (and cheaper) to launch our own agent from GH webhooks and trigger `/code-review`..
We've had great success with a custom gh action on a self hosted runner. It sets up codex and all the same stuff a dev has on their machine:
So dbs, seed files, etc, and then just a review prompt and some skills and a few scripts.
It triggers on a new published PR or if tagged @bot-name. It's way better than the other services we've tried (which was admittedly ages ago) because you can get it to work exactly how you want it and it's dead simple.
With Codex, for the time being at least, you can use a subscription, so it's basically free if you're already paying for one. But on API pricing it's still not too bad as it doesn't produce that much output.
I set this all up in about half a day. Codex does 99% of the heavy lifting.
this: gh actions using an opencode harness (so we can use cheap models through OpenCode Zen). With the right model, and the right, scoped access, it's just a prompt away.
bugbot was costing us $1k per month for a small team. EVEN IF we decide to go with a paid or say something like GPT Sol for our custom harness it'll cost us maybe $100. We've already implemented all of bugbot's interesting features like cross-PR memory.
the cool thing is we can leverage this for things that bugbot cannot do. Just today I added a security-reviewer action reusing almost all of the code that we wrote for the code reviewer.
Depends on model, promt and amount of agents that review. I found that default code-review in Claude code is pretty mid for finding all issues especially if you downgrade model and effort.
During my evals on my own 3 agents that review in parallel they are much more cost efficient and cover 75% issues. Even on model like haiku it’s better score than default. Multiple passes or raising effort will cover 90+. And this was tested on large diff, about 3500 LoC changes
> the claude code review product originally mentioned it cost "$15-25 per review" (which is absurd)
If an engineer paid $150,000 a year reviews some code manually for about 30 minutes, the cost of their time is about $36.
in a vacuum, sure. the average pull request will have multiple commits once opened (i.e. to address feedback or perhaps it was opened too early), each triggering a review. it's very easy to rack up $100+ per pr (and an eng may open a dozen prs in a single day)
I've set up a claude code on a vm with a bunch of skills that can use git and the Azure Dev Ops MCP via a service user, the regular build agent pops a record in a db that serves as a queue/review and a bash script picks things up and fires off claude code in headless mode.
It's pretty effective and lets you do nice stuff like compare the code to the tickets, reply to comments and stuff like that.
Using a different lab's model to review the code to the one that wrote it. Claude model plans/codes, OpenAI model reviews, or vice versa, or DeepSeek or GLM or whatever.
I do it this way too. I think having the same lab's model review the code in a fresh session also helps, so I use both: I generally start a new Claude Code and a Codex and have them review the code with a focus on 3 things: 1. bugs, 2. cut corners, 3. code style.
This generally works for me. I think a solid spec is the key for the models to review against. I ask them to follow the spec, and if the spec doesn't say, follow first principles.
Also, if there is a user experience change, they need to notify me.
How about your pattern?
[flagged]
[flagged]
[flagged]
Poorly I'm afraid, given how many new bugs make their way to the claude code CLI every update.
What I’m more worried about is feature identification and how to monitor them. How can you get the product to the next level?
Honestly I’m not worried about code anymore in my codebase I’ve not had bug problems for a while and incidents get fixed automatically.
I monitor new features with an agent who checks recordings.
Most actually good features need still 20-30 iterations to design the userflow to actually be interactive, but then there is also another challenge to translate usage patterns into features and how to present functionality with the least amount of friction as possible.
How to efficiently run monitors without wasting your token spends. A best 'monitor' best practice would be nice.
Still this article was very helpful and gave me a lot of new ideas.
That explains why the UI keeps changing.
How is this a proper product? Every time you open the desktop app the buttons may be on a completely random place
The UI has improved substantially over the past six months. New capabilities are prominently featured and less-used ones are de-emphasized or removed. Every product goes through this type of evolution. The difference with Claude Code is that at the speed they iterate, it happens a lot faster and is therefore more noticeable.
You can call it however you want, but in the end it is just poor engineering. Imaging building a bridge that needs construction every week
6 replies →
I use Claude Code for code review, and honestly it catches more than human reviewers have in my experience. It's been a big help.
It makes sense for Anthropic to pitch "more code = more better" but as anyone who's been in this game long enough knows, "less is more" often enough. And good luck coercing an AI coding agent to solve a problem by removing things.
Personally, I find Claude's degenerate urge to just spew uncontrollable amounts of code quite counterproductive. Yes I get it, you get paid by the token. I'd like to see more reasoning, more understanding, and less generating.
Truth is, we are still very very far from actual AI software engineering. Some people are working on useful tooling, and (predictably, like anything good and useful) it is taking a long time. LLMs are great, and hopefully they will help us accelerate building the tools that bring this profession to the next level.
30k PRs in a month (or whatever) is not the brag they make it out to be.
I think this is the future. Im more bullish than bearish. The only issue is that eventually human verification can no longer be possible and we will have to live under AI-verification assumptions....
We're already there with our pipelines and suspect a few more out there more concerned with measuring outcomes at this point.
They are in for a surprise when they become publicly traded and have auditors requiring actual humans signing off on any code that touches financial data.
You're in for a surprise if you read the post.
The post explicitly says they use codeowners to require a human in the loop for billing related code.
This isn’t just billing related code. If they tell investors they have X active users, for example, the code that tracks active users is subject to the same standard.
I am happy people are testing these things out. I am also just going to wait a bit longer before really adopting things like this, I don't really trust it yet. A year ago this looked very different, and next year my guess is things will look different as well.
Crossing the Chasm...
https://en.wikipedia.org/wiki/Crossing_the_Chasm#/media/File...
Yeah, the software factory idea has not crossed the chasm for me yet.
> At Anthropic, our jobs have moved from reading every diff to the decisions machines are bad at:
> ...Architecture that sets a pattern others will copy...
I'm happy that even Anthropic agrees this is best left to humans.
> Code review does two jobs: 1. Find and fix issues before they ship 2. Build shared context and codebase understanding across the team
So code review has become newspeak for “testing” and “education”?
Well not really testing. We the users are the testers and it falls to lucky us to really discover and find the errors, edge cases and security issues. The consequence of the rush to continually release…
Is the AI advance in the field of software testing nothing more than “now an AI agent can do a code review?”.
I built this PR Manager to help me review the PRs using Coding Agents. It sits on menu bar on macOS and has various lifecycle actions of PRs. feedbacks are welcome.
Github Repo: https://github.com/vs4vijay/prx
30 thousand changes merged per day? no. like just no. what are we doing? I'm trying to understand
There was a post some months back called "Lines of Code Got a Better Publicist" or something along those lines. This is just more of that. I am not impressed by 30 thousand "changes" per day in the slightest without significant added context. When I logged in and prompted Claude today, do I feel like I got +600,000 change value from when I logged in a month ago?
I’m horrified by it. There is no possible justification for it but it does explain their buggy products.
I, too, can write and merge 30000 changes per day. Even 300000 if I've up to.
Of course, just like "line of code" is a worthless metric, "change merge per day" is;
(automerged PR, JOB IS DONE)
This might be one of those things were everyone on the team takes a 30 minute lunch instead of 60 and just nobody clocks out to get the 1 hour extra over time.. And everyone just kinda ";) ;) ;) Working from our desk while eating lunch ;) ;) ;)"
Absolutely, at this rate most software would be complete in a few hours. I may lack imagination but wtf are you changing? Endlessly refactoring? That’s spinning your wheels renamed as productivity.
lol right? So youre telling me they shipping 150,000 features this week? Or, maybe, they whack-a-moled 149,995 perf or bug regressions and shipping 5 new things. :shrug:
The premise is clear, is one agent can do what developer can, but quicker, why not speed up the whole workflow to match it?
Have you used Claude Code?
Have you seen the benefit of the churn of 1000 PRs a day?
2 replies →
Right, but why do things this fast to begin with, especially for a product that's already publicly available?
I don't want to use a product that shifts under me as I use it several times a day. I want a product that actually moves slowly and works very reliably.
As an end user: why would I care about how many hundreds of releases a month are being pushed if maybe 1% of them have features I care about?
Speed is not the product.
More code output isn't a metric of success per se. Goodhart's law.
Humans don't operate at AI speeds. We're making the car go from 0 to 5000 mph in 1 second and the drivers are turning to goo in the seat.
1 reply →
Genuine question I’ve been wanting to ask folks working on different types of things: how are your teams doing agent-assisted code reviews? Not automated reviews, but what techniques and tools are humans using to keep up with the larger volume of PRs that seem to trend larger and larger?
I think a lot of people like myself still want to feel in the loop but I don’t have the right tools to stay on top of the (generally good, productive, and still thoughtful) contributions from peers anymore.
Using agents for a first pass is a good way to speed up the review cycle. This way contributors can get near immediate feedback before a human needs to weigh in. This doesn’t replace the human review, but augments it. Each time your agent misses something, update your code review guidelines.
Remind me to invest more in the clouds they are burning those tokens in.
If their only metrics are revert/incident rate then they're missing the long-term, cumulative impact of compounding complexity due to poor architectural decisions.
My thought as well, I think this approach only works for them because:
* They can afford frequent CC incidents since their value is having the best models, and 95%+ uptime is still OK-ish.
* The core functionality of CC (the harness "agentic loop") is simple, an if that works then for most purposes CC works.
* CC just stores a bunch of config and history files, there isn't any financial or irreversible safety loss if they mess it up once in a while.
Once you work on something where a minor slipup can result in a huge real world mess, then I don't see how their approach can work.
Anthropic is doing some fascinating work. However, I find their communication style frustrating. It’s clear to me their employees have near unlimited token budgets. When Anthropic team members share their internal processes, I feel they don’t appreciate the constraints that many of us face in our work with budgets.
I don't understand how their work can be considered fascinating if an insight into their internal processes shows that they are completely detached from reality. This would mean that their work output is also detached from reality. "Fascinating" isn't the word for this.
I use Claude Code every day. And I find the product to work smoothly, and effectively. From the user's perspective, that's good software. I'm impressed with what they've built. Truly. They are using a novel approach to write good software. That's fascinating. TBH this community has lost touch with reality. AI is making people lose their minds.
Its unclear to me how at least Anthropic hasn't been doing what amounts to recursive self improvement for a while now.
They heavily use Claude for both code generation and code review of Claude's own code and related tooling. Is that not functionally Claude running recursive loops to improve the next version of Claude?
> Claude Code lives in its own repository, separate from the monorepo
I have some bad news about that "monorepo" designation.
That's not news though. In fact almost no company beyond maybe 10 people that has a large central repository has no other smaller repos with other smaller production projects. They still always call it monorepo, because it approximately still behaves like one.
Tokens are also a scarce resource for some of us:
> THE SCARCE RESOURCE: HUMAN TIME AND ATTENTION
This workflow may not be accessible to token poor teams. Though I’m not all the way through the post, so I don’t know if token scarcity is even acknowledged.
I built a library to have a way to check Claude Code's work against API docs, and asked it to write many tests on top, but even then I found Claude Code would paraphrase and drift from the docs. I also found that it passed its own smoke tests but failed when testing against the real APIs, so reviewing Claude Code's code is tricky.
I simply don't want to review AI code in depth.
I used to do it but now I only review code organization and structure.
Mostly I define the product behavior I want and discuss the technical implementation beforehand.
Once that's in a spec I let it build in my durable implementation engine. Automated verification, review rounds from different agent types across different models, issues triaged into meaningful clusters and flagged for false positives before finally being fixed.
That loops until the code is up to spec.
There is no special secret, LLMs need repetition and multiple passes.
They'll one shot bigger and bigger things in their default harnesses but they'll be filled with issues no matter what.
The answer is that AI tools are not easy to use. It took me quite a long time to learn actually.
Eventual productization will probably make that effort meaningless but hey. whatever. I'm too old to sell my ass for cash too.
I think this is exactly why the current "attack of the vibe-clones" will fail.
Vibe-coding complex software can work, but it's going to take a really disciplined team to manage this properly, like in this article.
I'm astounded that I kinda ended up building the same agentic workflow they are talking about in this article and distributed it amongst the org (eng and product/designers).
On the other hand, I would have expected Anthropic to have a drastically more elegant SDLC.
I think they must be using the Fable or maybe more capable model for internal use purposes.
FWIW, Pangram identifies several sections as 100% AI-written.
I would have been surprised were this not the case
Interested to know how they handle ordering changes. Sounds like with that many changes, you're bound to hit lots of conflicts.
30k commits per day on a console app?? Are these all one-line changes?
I've written a harness myself and fair enough, it's more complicated than you might expect.. but surely at this point the claude CLI is pretty stable, and this must mainly be a ridiculous amount of test churn.
Or maybe their team of clankers is just creating busy-work to make the humans think they're essential. That's not unheard of in the human behaviours that they're trained on.
It's pretty bad, isn't it? I can't understand why that would need so many changes. It's like people/agents frantically working to look productive without any thought on the output or overall architecture. This is not something that should be copied or admired.
Absolutely.. If I had a project with 30k commits per day (or even per month!) I'd be keeping that secret, not blogging about it!
And yet they can't fix the janky bugs when you scroll or show/hide chain of though, commands executed, configuring plug-ins from the options menu etc.
The furious pace of Ux “optimisation” that we see across the board (and especially at AI companies) is a hidden anti pattern. The time and frustration of learning a new ux is an acceptable burden when it frames real new capabilities or workflows. It’s not good to rework your ux user experience every week just because you can.
Not only that, but AI is categorically terrible at ux review. I don’t mean aesthetics, I mean attention flow, workflow in general and what the user anticipates as a logical path to feature discovery and exploration. Increasingly, features are hidden behind workflows that meet a single use case but fail to be universally logical or in many cases, even accessible at all outside of the test case.
So, people… use AI in code review. It’s actually pretty great at it if you do it right, and you can ship better software with fewer bugs than ever before. But please, don’t pressure your ux guy to “10x” with AI. In fact, now that you are “10x”ing your development cycle, you are probably going to need to hire -more- ux developers if you don’t want your product to feel like slop even if it’s gold under the hood.
I wonder how they maintain the quality of tests for such a massive codebase and frequent commits.
That's the trick: they don't
90% ?
A random twitter said that, 6 month ago, it was 100% !
DECLINE ! SELL THE STOCK OPTIONS ! (wait, there are none, confused!)
The writing in that blog has the distinct whiff of Claudish to me!
Claude Code is great, but do you really push that much (new) code every day?
I hope my competitors do code review this way.
That's a shockingly low number.
1000/day in what sounds like a single repo is a low number?
I am not sure what it is you are saying. What I am saying: I would have supposed that Claude writes 99.x% of Claude's Code code, and that ~no code at Anthropic is written by humans anymore.
1 reply →
No. 90% sounds like a low number. That implies something else is writing 10% of the code? People? Other models? What on earth is writing that other 10%?
The 30,000 number in the blog is the number of changes landing per day, not per month!
to achieve what?
Honestly not sure that there are that many effective changes a day given how little complexity there is to their public facing product.
Sorry, but how the hell does a TUI harness have 30,000 changes _a day_?!
The 30,000 number is across their entire monorepo, which they say in the article Claude Code does not live in.
everything is shipped with auto reviews behind a flag. then you need a human approver to launch
good model
this article seems written by opus 5
I'm confused because I don't imagine the Claude Code team as particularly productive. They certainly don't standout compared to pre-AI times.
If the entire body of this article isn't just an "LGTM" then I know they're lying.
They use Codex?
Title is: How the Claude Code team does code review
spoiler: they don't
I don't care how they do it.
I vibe coded my harness that is superior to CC in functionality, and it still doesn't flicker.
I don't need to hear their bullshit.
Anthropic's solution to anything is to simply burn more tokens. Because of course.
Anyway, flagging as misinformation.
[flagged]
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
[flagged]
[flagged]
[dead]
Do they still have humans left on the team? /s
tl;dr: We don't, claude does. Make of that what you will.
We have to test many different api's and models as we have over 600+ in our harness now, Sonnet 5.5 via cloud code provides the highest quality results.
Over the last few years we have added mcp tools, memory graph, md files and skills, automated everything to the point that on desktop the system runs itself, besides proceed/approved/explain.
We have 100's of PRs/jobs/projects and Claude builds down the list automated and only asks a few questions point form if needed.
Very impressive now, compared to 6 months ago.
You can view it if interested: www.vekmem.com
Very impressive indeed https://imgur.com/a/xbwdkWc