Comment by LPisGood
15 hours ago
My coworkers continue to dump hundreds of lines of AI documentation in every PR and every other line of code has between one and ten lines of AI generated comments, talking about the real unlock and how things are byte for byte identical on the load bearing path or how the acceptance ladder is misleading.
Features are coming out and metrics are improving, but we’re basically in a post readability code base, with the occasional performative comment about a variable name.
I don’t really know how to address this situation or if it needs addressed. I certainly don’t read the long-winded AI comments or the AI documentation, but perhaps it’s useful for the AI on its next pass.
My "favorite" Claudism is when I critique its work and ask it to remove some unnecessary part of the design -- and then the diff has more green than red because it added comments about why the code is no longer there -- the code that was never in the mainline and never asked for!
Oh god this has been driving me nuts since Opus 5 landed. Every docblock is filled with long-winded jargon explaining why this design is superior to some other design, which never existed as far as any dev who might read that comment is concerned.
This has been a thing for long while, on codex too.
You ask it to do something, then tell it to do something in a different way, then it assumes it needs to do the refactor in a backward compatible way, or creates migrations for it etc.
4 replies →
Oh no, you said "landed"! It's dug into our brains!
14 replies →
There needs to be a pithy name for this. I have been calling it context leak.
1 reply →
It's not just claude, all AI is unable to produce something concise. On the surface everything looks 'good' whether code or prose, but then if you dig a bit, try and understand the whole text you quickly realise that 80% of it is unecessary and the whole thing could have been re-worded/re-coded into something a fraction of its size and complexity.
I asked Sol to reduce the length of some documentation we had by making it more concise. It came back after 20 minutes of work, did a line count and was aghast that the line count had somehow increased...
It's not that it fundamentally isn't able to produce something concise, it's that the business model of the companies developing these models rests on selling tokens...
I have a theory that AI code looks good because you never subsequently come up with your own alternative.
4 replies →
I have to ask Claude to compact the comments every time, and I give specific criteria for it. Never ever reiterate what’s in the code, never mention decisions not made, never mention the conversation, etc etc.
Even then it is conservative. For the love of God, compact the comments.
Comments become a huge maintenance burden, especially in the age of AI. They just grow and grow, and then mislead the AI later on.
3 replies →
Oh Jesus this. I've tried to include rules that tell Claude to only include relevant, evergreen comments but it's to no avail.
I also love how it'll build local plans with phases, tasks, or decisions, then reference those numbers in those same useless comments.
Or when it references "item 06" from some plan you wrote as if it had any relevance to you whatsoever.
Yeah this is awful. Every codebase becomes a graveyard of references to ideas or behaviors that were barely considered. It's probably also a compounding source of context poisoning when a minority of the comments/documentation are about how the current code actually works.
It also likes to spew references to documents that are not, and never have been, in the repo. So if you're not careful you'l have comments all over your codebase saying things like: foo() - Perform foo action as documented in PRIVATE_INTERNAL.doc
1 reply →
shudder
I wish they trained the bots to be a notch more relaxed and less hysterical. Less is more.
But maybe that's just a consequence of the RL training being essentially AI torture to make them do what we want.
This is a common problem, and I don't get why LLMs have not been tuned to stop this nonsense. It is writing comments as if the audience is you, the user in the session, while obviously code comments are meant for future readers.
Follow the tokens
Have you considered talking about it? You're in a professional environment collectively working in a new way with a group of people. It's up to somebody to have opinions about what does and doesn't suck. If you silently go along and don't say anything you're dooming yourself and all of us to a lifetime of this garbage.
Fighting the ocean is futile
It's not the ocean, it's the poster's own team. A simple "AI comments suck" in a sprint retro would be trivially easy and would at least start the conversation.
20 replies →
So is completely eliminating litter, but I still pick it up when I pass it.
2 replies →
Literally pissing in an ocean of piss.
2 replies →
Yes, talking about it repeatedly has been my process. Individual PRs have been changed but the 4 the next day look the same. The typical response is that they did change the (massive and wordy) PR overview from what AI said, even if it was obviously a minimal change at best. I can’t really argue against lying and going through and picking out every LLMism is not so productive in my opinion.
I don’t have time to go through and flag everything or even read the thousands of lines of code changes that are happening.
I've tried doing this, politely pushing back on problems happening from AI use, and it led to a not-so-subtle implication from my manager that I need to STFU or I'll be out of here (Cisco).
No, if he speaks against it he gets labeled "anti-AI" and laid off or not promoted. If he stays silent he retains his job.
Corporate Capitalism's main innovation was virtualizing feudalism. Now anyone with a bit of cash flow can be a feudal lord with peasants to do his will. No need to maintain land or fight over it with other feudal lords. There's still fighting, naturally, but now the game isn't zero sum.
The serfs don't get any real say in this model. At best you can bend the lord's ear if you're sufficiently trusted an advisor. But part of the reason you're trusted is to be trusted not to treat the lord like he's too much of an idiot.
A huge problem is that capitalism rewards cash flows and accumulated capital so powerfully that lots of other things just don't matter, potentially for a very long time. Look at GitHub. This leads to immensely extended fuck around phases and "find out" looks like IBM (fossilization and bizarre holdings corporation / dead company parking lot) just as often as it looks like Enron.
> The serfs don't get any real say in this model
I am a programmer but also in management, and if I knew my team thought like that I would be horrified. We absolutely care about what the team members want , like, hate etc. I for once would love someone to bring up stuff like this to me. As long as they are as open minded about things as they would like me to be, there is no issue. But it’s a serious issue to pretend everything is fine while thinking like you say.
1 reply →
My team uses a Claude Code hook that blocks any comment more than 2 lines long, and when tripped it encourages the agent to rewrite the comment more concisely and focus only on the "why" not the "what" of the code. I've found this extremely useful for code reviews.
Two very useful directives to give AI when it comes to documentation:
1) Document what's there, not the diff. Documentation of how code was removed or changed to fix a bug or add a feature is not useful and difficult to maintain; documentation should explain how code works now.
2) Documentation should live close to the source as possible. Prefer line based comments and standardized function documentation. Top-level sweeping architectural essays are not maintainable for every change.
The last will depend on your codebase. It CAN be very useful to have a human-readable spec documented for the entire program and have it updated when anything changes. But the key is again, you're CHANGING it every time. If you add a whole new disconnected documentation file it should set off alarm bells; nothing in one system is truly disconnected.
> Document what's there, not the diff
We recently added a similar thing to our style guide, It’s astonishing to me that we have to spell this out, that something as obvious as this needs to be explained to LLM’s at all. They’re supposed to be exceeding human intelligence, at least at things like programming, but can’t understand basic things like what code comments are.
My theory (which might be completely wrong) is that models do this because it improves quality for vibe coders.
When vibe coding the content of user prompts is ground truth and the only way any human thought affect the code base. So if the vibe coder says "do X not Y", recording int comments that "we shouldn't do Y" is important. It ensures that the agent doesn't accidentally decide to do Y tomorrow, which would frustrate the vibe coder who'd feel that the agent doesn't "remember" what it was told yesterday.
So for people who look at the code the comments are obvious and completely superfluous, but for the vibe coder it's a way to ensure that their tiny (relative to the size of the code base) input is not forgotten.
> It’s astonishing to me that we have to spell this out, that something as obvious as this needs to be explained to LLM’s at all.
Hehe. Yeah, that tendency of LLMs to document "the story" of the code instead of its current purpose (or non-obvious implementation details) is a pet peeve of mine too. I've added a slew of guidelines to try to sway Claude to not do this, but it still does it often.
At the same time, it feels like something to be expected to have this "failure mode". The model has its context to work on, and what is on its context if not the conversation you've been having (and its internal monologue) and the files it has read? It makes sense that it references the story on its text generations, because that behavior is usually a good thing for an LLM to do. Otherwise, what would it generate? If it generated things that had nothing to do with the conversation in its context, in many cases those things would be seen as "hallucinations", and they'd tend to be RLHF'ed out. So the models that we end up having are the ones that have been reinforced to be most "contextually relevant" and less "hallucinatory".
I might be completely wrong on that of course. It's just my intuitive reasoning of why this seems to be such a prevalent behavior.
Most models are trained to be as "helpful" as possible which may work for a chatbot but not for code.
It's a bit weird, because that seems like something that approximately the same in every code base, so should be relatively easy to train generically.
>They’re supposed to be exceeding human intelligence, at least at things like programming
This perception is a good part of why this market is irrational. LLM's aren't "intelligent". They do not reason, they are a very fancy kitbash of whatever it trains on.
Ad yeah, I'm not surprised that a lot of documentation on every bit of readable code online is awful. "Document the diff" sounds like an anti-pattern learned from people with an incentive to get as many PR's submmitted as possible, not make the most friendly documentation for people maintaining a project.
And the tests. Oh god the tests. Personal recent favorite: I asked for some changes to a Dockerfile, which it did ably, and then promptly tested by writing a pytest module that traversed up to the root, read the Dockerfile, and checked that the added lines were present.
Try property based testing perhaps.
Code-generating robots are pretty bad at property-based testing, in my experience. They can do it but they still need a lot of hand-holding. They often regress to writing a mirror implementation as the oracle and trying to enumerate a fixed set of examples they find meaningful.
I'm generally happy with the tests I ask it for, some of which are PBT. It's just the insistence upon memorializing every single change with a test. Maybe encouraging / forcing PBT will dissuade it?
I’ve heard of this, but I’m not really sure at all how to even get started. Are there any good guides out there?
1 reply →
I regret this but at some point I stopped reading generated tests. It feels pointless when our test files are already tens of thousands of lines of — at best — tautological slip which says that the codes does what it does.
I told someone this week, who (or whose AI) chose to do a problem the hard way that it's usually a bad sign if you need more comments than code to solve a problem, and then suggested a couple lines of code that accomplished the same thing and used, are you sitting down? MEANINGFUL VARIABLE NAMES to document the purpose of each calculation.
I wonder if I can get a MacArthur grant for this epiphany...
Be the change you want to see :)
I've created myself a pre-commit harness hook to explicitly discard superfluous or too lengthy comments. Within code-review I also added comment review as blockers
I have five enforcement mechanisms: 1000 line max edit, PR comment character limits (get to the point of your description), ISO 24495 conformance check, and enforced code line citation that must exist, be a function declaration for the start of all paragraphs and inline commentary must be three lines or less and inline comments contribute max 10% of the PR. Fail any of these, automatic PR denial with no human intervention.
This sound pretty good, but every single attempt to put an actual character limit meets incredible resistance on my team. ISO 24495 looks interesting, how do you enforce that? Do you have some agent?
Table of words LLM generated, Binary Searched on the output going forward and local skill/CLAUDE.md line with instruction to conform. The comparison then is pretty fast due to the word limitation rules. Also standardized PR format so a bunch of what a dev would usually communicate is just a series of checkboxes and a place for adding an optional link for "additional discussion" on slack as the escape hatch for people who like to waffle.
> ISO 24495 conformance check
How do you enforce that?
Binary Searched against a table of words. It's rough at first, but once you add contextual exceptions the false positives calm down. Also the CLAUDE.md file has an instruction to conform before even submitting the PR and there's a skill too for it to self iterate.
Prune the comments? Instruct the LLM to print less comments (this one is genuinely hard though). What's really happening is that you don't have a strong enough review process (or a code standards process) to offset this. The one issue I see with this is that your team is almost certainly _NOT_ doing any kind of code review (especially if they're leaving comments like that). The other problem is that excessive comments actually harm LLM output, I've done tons of A/B testing, and pruning comments actually helps LLMs spot bugs, among other things.
I think your last sentence is getting close to the truth. You're no longer the audience for those descriptions. Other robots are.
I'm not saying that's good or bad because I don't know, but I think that's the idea of dumping all that junk into PR descriptions.
However, annoyingly, we still need to review those descriptions very closely, because the robots are trained to put a lot of weight into things they read in the documentation. And they tend tospresent loose speculation as fact. They often end up documenting some assumption that isn't true, then end up writing code as if it were.
Even worse, in a brownfield codebase that was once fairly light with comments, that's now being subject to these modifications, the insane amounts of commentary around the parts newly touched by AI lead to an excessive emphasis on those parts, for both human and AI readers (who think, well if this one part is commented so thoroughly, it must be unusually subtle)
Honestly, if you saved a ton of hours with the model coding for you, at least give me 30 minutes of your own words, show me you know what you're shipping, if you can't do that, then I don't know if I want to approve the PR. My first job we always did peer review in a meeting room when a PR looked a little too much, you can't exactly bring in GPT into a meeting so its a good time to ask simple questions about the change to ensure you understand it just as much as they do.
(usually) You're not in position of power to effectively keep that position. As comments aroiund - standing against will mark you as anti-ai luddite and will now end well for you, not AI-spammer.
> you can't exactly bring in GPT into a meeting
They totally gotta be doing that at OpenAI. Meeting invitees: You, co-workers, GPT 5.6.
Well yeah, they'll keep burning the VC bucks.
It's like people didn't realise that it was unmaintainable before and now we have a new level of unmaintainability. The insane amount of code produced means it's only maintainable with AI.
Congratulations: now only AI can iterate on your codebase!
Even the AI degrades as the codebase gets shittier
Degrades? Nonsense. Sounds like you need the latest model and more tokens.
That’s probably not completely true, but it’s approximately true.
It's a code review, right?
Give feedback that about the docs and block merging till the issue is resolved.
This sounds easy in principle, but a half dozen of these sort sorts of massive PR’s per week is basically untenable. I’m not gonna read the hundreds of lines of added documentation to decide if they’re correct or not. The price of generating new words is just so much higher than the price of evaluating it that I can’t be bothered.
Just hit "deny." I've been training my co-workers that the AI-isms their "AI Assisted coding" do that some shit just isn't acceptable. I leave a comment and hit deny. It also helps that I control the policies on the repo and they can't merge in with a denial from anyone. So it's either a fix it, or explain why your work isn't getting done. My manager is also 100% with this.
I've denied for poor branch names and commits from AI. I've denied for too verbose of comments from AI. I've denied for parts of the code base being touched that are not relevant to the case they are working on (login isn't broken, your case is to add a check box in the settings pane, remove the changes made to login).
Pre-AI I wasn't fine with PRs with multiple features and touching irrelevant areas of the code base. Why would I be fine with it because my co-workers got new toys? You want AI to refactor a part of the codebase? Fine. Separate branch, new PR, and in the description, present an argument for it. Don't shoe horn it into something else. Also, I'm not obligated to hit approved on shit.
I may also be a bit privileged because I can be a pain in the ass to whole team. I may be slower, but I've got the numbers, my code is creating way less bugs then my "fast more efficient" co-workers.
15 replies →
> but a half dozen of these sort sorts of massive PR’s per week is basically untenable.
Actually, rejecting them is precisely what will make them easy.
"Sorry, the comments are so bad I'm stopping here. Please fix them and then I'll resume the code review."
You're giving everybody (including yourself) more work by:
1. Reviewing the code (even if you skip the documentation).
2. Letting too many abstruse comments in which everyone in the team will have to read.
3. Allowing the behavior to continue.
Become the bottleneck so the team can talk about it. If they decide this shouldn't be a blocker, just declare you won't review the comments going forward.
13 replies →
You can't keep up with the slop. And before you finish a first pass read on the wall of diff, another AI sloperator on the team has hit approve and the PR is merged.
At that point maybe go looking for a company with a better work environment.
Exactly this. Even if I spend a bunch of time requesting a review — and our team does respect each other enough to at least nominally respond to comments before merging — the update itself will be another thousand line diff from the original that requires again the same level of review or I just accept that it looks fine.
I have my agent write up a summary of the diffs that land each day in my org. If there is something you interesting I’ll ask for an html explainer with code pointers and scan the code in parallel.
I wouldn’t say “post reading code” but it’s definitely trending in that direction.
I’d rather the agents put jumbo verbose descriptions in the PR description than in code comments TBH.
You forgot the smoke tests that passed.
So many smoke tests in every PR i make with AI. Not that i mind, I just find that phrase funny.
Am I the only one who's had Claude almost systematically remove human-written comments?
It might be touching one line of actual code in a file, and take advantage of it to remove 20+ lines of actual useful comments.
Everybody is talking about the opposite, so I'm wondering if this is rare.
Is this in one of the skills or CLAUDE.md? This was happening in our codebase, but turned out it was interpreting an instruction to not add "what" comments as license to strip out comments.
Sometimes I have luck interrogating Claude on why it did something. It'll either point to a skill or agent file with the culprit, or it'll respond with some vapid nonsense and apologize.
I've definitely seen this. One of my least favorite parts of developing with AI is when I add print statements or small changes and the LLM removes them in the process of doing the next thing. I want to work _with_ the AI, not have it stomp all over my code.
Do you think it's actively deciding to remove them or just not noticing you added them and then overwriting?
I ask because when I started informing it I made personal edits, it stopped doing this kind of thing so often and let me work _with_ it more.
My saved prompt now says never to assume a file has not been edited since the last time it was read between prompts.
Most ai output is meant for other ai's to read, in my experience. The humans job is to compress it for humans
I think at this point all the info added by AI which certainly would be too much to read for every PR, it just serves the purpose of context for the next action. Which it could be good or bad depending on how big of a window of context you are working on
Honestly, I’ve stopped caring about code readability for a few months now. I want the code readable _to the agent_ not so much to me.
I don’t trust it with code anyway - every feature needs comprehensive test, and then a live deploy on a real working test system before it is approved - I mostly measure success with - after deployment is it doing what it’s supposed to be doing. It’s like “helping another team managing their work stream” experience rather than coding yourself.
Funny enough models seem to have personalities and the dis on each other - when I had an opus orchestrator dispatching fable workers, they would comment on how “unreliable” it was and it had “evidence to prove it” and fable thinks opus is too rigid and needs more hand holding… it really starts to feel like managing team egos and verifying work. And I code scan mostly to just spot check if it’s not doing anything super stupid. But my goal is to make sure anything shipped is easy to change and fix, and every mistake has a test behind it so it doesn’t happen again.
I ship more problems, but they get discovered and fixed quicker. Before they reach prod of course. And from time to time you do reorganisation and refactoring passes where I brainstorm how things could have been better with the help of evidence- chat sessions, tests, bugs etc.
It feels less like rigorous engineering and more like gentle gardening. Or like “project management” not “coding”.
Honestly given my age now I’m fine with that. Have enough “hard” projects under my belt (ORMs, sql parsers, etc) that I don’t feel I need to prove anything to anybody, but I don’t think that’s even relevant- the velocity change is … I guess around 5-10x for me - with provable metrics, so I try not to lent the good old days but figure out how I can now live in this brave new world and be happy with my work.
> I’ve stopped caring about code readability for a few months now. […] I don’t trust it with code anyway
If you don’t trust it with code, surely you need the code to be readable so you can understand what it is writing?
I'm looking forward to when AI labs focus more on conciseness of code and writing.
I dump AI output in PRs, because it ads context for the AI reviewer.
With one colleague, I was leaving PR comments and he would just put my feedback into the AI and paste its response. So I decided to cut out the middleman and now I just @cursor and tell it to trim unnecessarily long comments.
> I don’t really know how to address this situation or if it needs addressed.
My worry is that after several passes this compounds and starts introducing errors or biases, a bit like in the "telephone game" children play.
> My coworkers continue to dump hundreds of lines of AI documentation in every PR and every other line of code has between one and ten lines of AI generated comments, talking about the real unlock and how things are byte for byte identical on the load bearing path or how the acceptance ladder is misleading.
They're just helping you understand the whole picture!
> Features are coming out and metrics are improving, but we’re basically in a post readability code base, with the occasional performative comment about a variable name.
You futilely grasp for control and it eludes you. The Way is to ride the tides of life, move with the forces that shape you. Your code base is in the hands of the Machines now.
Just wait until you see vibe contracts, vibe requirements and vibe legal documents
I'm... Actually fine with that. In one direction. I use AI to fill in forma and usually it has really good pointers. I do not trust it to do it itself, but it does simplify things quite a bit.
I always as for CONCISE documentation.
still get walls of text sometimes
You use AI to summarise it! That's the way to go lol
I address it with AI.
Write REVIEW.md.
I have CC check itself pretty well.
I also put into agent/claude/review instructions to write using simple English skill and humanizer skill. Then not to write redundant comments.
It’s not perfect but definitely catches lots of slop.
It's actually insanely difficult to get LLMs not to produce comments. Even with explicit "NEVER LEAVE ANY COMMENTS WHATSOEVER", they still do, across basically all providers.
No problems with GPT-5.6 Sol here. I have an agent file that says, among other things, only to document purpose and intent, not just what the code does, and not write obvious comments. It's been so effective that it often doesn't comment anything at all, including some stuff that's so niche that it must be explained carefully. As a result, I've had to pull back a bit and tell it explicitly which areas to actually add comments for.
> perhaps it’s useful for the AI on its next pass
Yes, that's the entire point. And it is extremely useful. Why wouldn't I want this?
Is it really, or do you just think so and it could actually mislead you and your LLM the next time? In my experience, the information in the comments tends to be quite redundant, often even with other comments in the same file
Often even wrong.
[flagged]