Comment by rcxdude
6 hours ago
Hmmm, I don't think that's necessarily true. Often times once I have witnessed a bug, I have found it just by reading through the code with the behaviour of the bug in mind.
For LLMs, this is likely to be disproportionately effective as well: especially because they don't really build up a persistent view of the codebase, they're generally re-reading it each session, and they tend to be surprisingly good at predicting the behaviour of code.
(That said, knowing where and how to gather more evidence to make things clearer is a pretty core skill in troubleshooting, so it's generally good advice anyhow)
Also Claude Code is very good at writing small scripts/on-off test cases to confirm bugs, so I wouldn't even say the initial premise is correct.
That's AI getting an example of replicating the state that shows the bug which is exactly what I'm talking about. It does that far more than humans do, and it's ace. That's how you should be debugging a system - replicate the issue, understand why it breaks in that given state, and then make a code change to fix it.
Sometimes you can do that mentally and fix the code. Often your fix will be right especially in a relatively simple part of the code. However, equally often you'll fix a different problem (or something that wasn't a problem at all), and the original bug will remain but you'll believe you corrected the issue. This is why you should always replicate a bug to understand it, and why you should always add a test whenever you fix a bug to prove you actually fixed it as well as preventing future regressions.