I include a rule in the initial PR prep that it runs. So there's a hard limit to the length of the comments and what type of comments it can write and they get flagged in the review. It also reads the comments and checks for consistency with the following code so they don't drift. And the reviewer is a completely different agent/different model.
So all of that happens before the manual review and usually catches a lot of the 3-5 line comments it inevitably adds.
I include a rule in the initial PR prep that it runs. So there's a hard limit to the length of the comments and what type of comments it can write and they get flagged in the review. It also reads the comments and checks for consistency with the following code so they don't drift. And the reviewer is a completely different agent/different model.
So all of that happens before the manual review and usually catches a lot of the 3-5 line comments it inevitably adds.
I just delete them most of the time. But I keep my LLMs on a short leash. No 40000 line PRs.
You could probably add linting rules to your tool of choice and tell claude that your lint tool has to pass.
but then no human generated comments allowed either, right?
I'd do this by only running the linter locally and using this as a custom rule for the LLM, but I need to think about it a bit harder.
4 replies →
Most lint tools allow exceptions if you add a special marker. I think even allow custom user exceptions.
E999: human generated comment :-)
2 replies →