Comment by svaha1728
4 days ago
I completely agree with the author's comment that code review is half-hearted and mostly broken. With agents, the bottleneck is really in reading code, not writing it. If everyone is just half-heartedly reviewing code, or using it as a soapbox for their individual preferences, using agents will completely fall apart as they can easily introduce serious security issues or performance hits.
Let's be honest, many of those can't be found by just 'reading' the code, you have to get your hands dirty and manually debug/or test the assumptions.
What’s not clear to me is how agents/AI written code solves the “half hearted review” problem.
People don’t like to do code reviews because it sucks. It’s tedious and boring.
I genuinely hope that we’re not giving up the fun parts of software, writing code, and in exchange getting a mountain of code to read and review instead.
Yes, this is what I'm fearing as well.
That we will end up just trying to review code, writing tests and some kind of specifications in natural language (which is very imprecise)
However, I can't see how this approach would ever scale to a larger project.
This is an attempt to change software development from a put out system to a factory system.
It seems to be working sadly. If people hated agile, just wait for the prompt/code review sweatshops.
Yeah, honestly what's currently missing from the marketplace is a better way to read all of the code, the diffs etc. that the LLMs output, like how do you review it properly and gain an understanding of the codebase, since you're the person writing a very very small part of it.
Or even to make sure that the humans left in the project actually read the code instead of just swiping next.
Isn't that the point of agents?
Assume we have excellent test coverage -- the AI can write the code and ensure get the feedback for it being secure / fast / etc.
And the AI can help us write the damn tests!
No, it can’t. Partially stems from the garbage the models were trained on.
Example anecdata but since we started having our devs heavily use agents we’ve had a resurgence of mostly dead vulnerabilities such as RCEs (CVE in 2019 for example) as well as a plethora of injection issues.
When asked how these made it in devs are responding with “I asked the LLM and it said it was secure. I even typed MAKE IT SECURE!”
If you don’t sufficiently understand something enough then you don’t know enough to call bs. In cases like this it doesn’t matter how many times the agent iterates.
To add to this: I’ve never been gaslighted more convincingly than by an LLM, ever. The arguments they make look so convincing. They can even naturally address specific questions and counter-arguments, while being completely wrong. This is particularly bad with security and crypto, which generally isn’t verified through testing (which only proves the presence of function, not the absence).
Saw Rich Hickey say this, that it is a known fact that tested code never has bugs.
On a more serious note: how could anyone possibly ever write meaningful tests without a deep understanding of the code that is being written?