Comment by abuani
3 hours ago
> Here’s a third one. In some of the code reviews I’ve encountered that AI gives a lot of feedback, it’s just providing noise
Something I've found fun is seeing how long it takes for an llm review tool to come back satisfied with a PR. Think 100 lines of code changed, nothing terribly significant, but also not trivial. I'll have a local Claude session setup to babysit the PR and wait for feedback, accept all the recommendations, push the change up and request a review. I cap the number of iterations at 10 just so I'm not blowing a stupid amount of money. I've yet to come up with a PR where the llm reviewer is satisfied with the changes and has _no feedback_.
So where's the reasonable cutoff point for llm based reviews?
Huh, I've had many times when `codex /review` comes back satisfied on the first shot, both with handwritten and LLM-assisted PRs.
We use only one round of LLM review, and have discussed as a team still assessing recommendations, not just accepting everything blindly. So somewhere in the (0,1] rounds of review. Sounds like our preferred ratio of human to LLM involvement is different than yours, though.