Comment by fzeroracer
6 days ago
You're massively burying the lede here with your statement of 'just take the PRs where the code is straightforwardly reviewable'. It's honestly such an odious statement that it makes me doubt your expertise in reviewing code and PRs.
A lot of code can not and will not be straightforwardly reviewable because it all depends on context. Using an LLM adds an additional layer of abstraction between you and the context, because now you have to untangle whether or not it accomplished the context you gave it.
I have no idea what you mean. Tptacek is correct. LLM does not add an additional layer because at the end of the day code is code. You read it and you can tell whether it does what you want because you were the person who gave the instructions. It is no different than reviewing the code written by a junior (who also does not add an additional layer of abstraction).
That's exactly what an additional layer is! If I outsource coding to someone else whether it's a junior engineer, an outside firm or an LLM that is an additional layer of context you need to understand. You need to figure out if the junior engineer grasped the problem set, if the firm understood the requirements or if the LLM actually generated decent code.