Comment by SkyBelow

This isn't just on code.

When I'm writing technical documentation, it keeps the explanations in. Same when writing non-technical documentation. When I was having it attempt to generate a Pathfinder 1e class for a Sword Dancer, it was leaving in notes about why it removes things I told it to remove/rework.

And it isn't just Claude. I've seen the same with GPT models, with Grok, with Deepseek. Each AI isn't quite the same with how it approaches this, but in every case they seem to have a strong bias to retaining information, even bad information that we want gone, so it is like they have a, dare I say, subconscious bias to retain the information. Putting a note in a comment or explaining why to not do something or something was undone is a good way to retain information while still achieving the goal (well, if you ignore the part about the human intention for the information to be gone).

This then weakens the AI in the future, as I find AI struggles with the more incorrect information. Sure, a comment saying "not X because Y" is less 'context damage' than a comment saying "X" (assuming X is wrong), but it is still a slight shift to X being present in context in some way. One off, AI's seem to perfectly handle this without issue. But after hundreds or thousands of cases build up? The attention mechanism seems unable to keep up and incorrect information flows it. This effectively creates a sort of vibe coding maximum size unless there is a human janitor cleaning up the bad information on the context stays nice and clean.

But this is all simply a feeling I get as I use AI to do different things and isn't at all backed up by any formal study.