Comment by ares623
7 days ago
What I’d really like to see is a company that completely does away with the code review step. Because if you think about it a code review is already being done by the invoker of the LLM. That way the velocity will actually be faster. It feels like at the moment most of the velocity is blocked by the outdated notion of code review.
It’s also why these tools feel great in green field projects, since these ones don’t typically have any code review in place (i.e. one dev going brrr)
We tried this as a bit of an experiment - not quite greenfield but a fork and modification of an existing project.
I was a solo dev going brrrr with no code review, using Cursor to make pretty sweeping changes and implement full features. I think because it was a fork I was able to go really fast because I could point the coding agent at existing patterns and say, "do this but different".
Occasionally I would test changes in staging but mostly would just promote them straight to production. I went on like that for ~3 months and the product ended up getting ~75k (daily!) users in that time with *no* outages or downtime (!!!)
It worked really well! Until I hit my scaling limit and we added more engineers to the team. A new hire on their second week on the job shipped a change that was also written with AI and caused a 7 hour outage before we realized something was wrong. I even *thoroughly reviewed the code* because I knew it was generated and I made a comment questioning the lines that caused the outage, but after we talked it over we decided it was fine and was only one tiny bit of a much larger change.
So I guess, AI will ship bugs but so will peer-reviewed code.
Nice. Sounds like a success? Was the experiment made permanent? If not, why not?
I realize this is likely being facetious, but just in case - code reviews are so much more than just 'check the syntax and style of the code'. They check the intention, check the actual functionality, find issues on the larger scale that LLMs literally can't.
Yes, PRs start piling up because devs can vibe code them faster than they can be competently reviewed. This is a problem with the vibe code process, not the code review process.
I was being half facetious, yes. But wouldn't the invoker of the LLM be already doing a review in that case? It just feels a bit redundant, to have engineer one do a code review of LLM's work, and then have engineer two do the same review.
The business cost with PRs isn't the first review but another developer, it's the number of iterations on a pull request due to defects and change requests. The way I am trying to promote the use of LLMs with more junior developers in my team (I am the CTO) is to use AI-assisted tools (we used Windsurf and recently switched to Github Copilot) for a first pass, e.g asking the agent for a review and catching potential defect before involving the human reviewer.
This doesn't mean the human reviewer will need to spend less time reviewing, but potentially this PR will be merged faster with on average a lower number of iterations and improved code quality.
I do have in my team some senior developers that are excellent, and it's very very rare I catch an issue in their PRs (maybe 1 out of 50). But I also have greener developers for who the ratio is way higher (like 8 or 9 out of 10) and this means repeated context switching for the reviewers.
I hope this is a joke...
Partially serious. I mean what’s the point of being able to open 100 PRs in one day if your coworkers can only reliably review 5 of them?
Not just that but PRs are about a lot more than syntactical correctness, it's also about design. Just this week we've had two PR catch bad design choices that came from a fundamental misunderstanding of the problem space. Even if the AI wrote perfect code the first time everytime it wouldn't matter in those cases.