Comment by t-writescode
23 days ago
Beyond mirroring the engineering practices that you yourself want to see other people perform, have you found any techniques to get people to … in short, do their job again? Understand context, understand what they did, why they did it, what they’re doing, etc.
The +/-2000 line MR was bad when humans wrote it. It’s way worse when the human didn’t even write or read it.
And just vomiting automated CodeRabbit talking points back and forth at each other feels equally harmful.
Are we really tolerating turning ourselves into LLM rubber stamps?
For me the trick was just leaning all the way into it. I had a residual idea that if someone sends me a 2000 line PR, 10 page design, etc., that this represents some concrete investment of time and effort that deserves my careful consideration. And it just doesn't anymore.
I have one project where there must be hundreds of pages of design proposals I have not read and will never read, because the author really likes having Claude generate complete design proposals based on incomplete understanding. So every week or two he sends me a new one, I spend 30 seconds skimming it, and then I tab back to Slack to ask him to explain.
I don't like working this way, but you know, I don't like doing rollouts either. It's certainly better than being a human rubberstamp.
That sounds like a fantastic way for a malicious actor or an unintentional prompt injection exploit to sneak into a codebase.
Perhaps you could explain how this is different from rubber stamping, if it’s just 30 seconds of reading.
Does the conversation you have reveal what they actually want?
And what about the 2000 line change? Does that get stamped after someone talks about the change but without deeply reading it?
Typically what happens is that we have a good conversation, we make progress towards figuring out what they should want to do and how they should try to do it, and the 2000 line PR or 10 page doc gets abandoned. I do read things in detail when I expect I might one day be convinced to approve them, but that fraction has plummeted from 95% to like 20% since January.
3 replies →
I tried to get our team to enforce a max PR size.
It worked for a while. There is a GitHub action that you can configure to fail if the PR is too large.
But then we started a new project and I didn’t add it right away since it’s a small team and I figured we could use the honor system.
Since then there have been lots of massive PRs but there’s not much willingness to go back to enforcing the rule because it might slow us down…
It’s frustrating.
Out of curiosity, with their current paradigm, what's the regression rate *and* how long do PRs stay in pending?