Comment by verdverm
1 day ago
> The rate at which we're churning out AI-generated code is going to make this worse
Not only this, a significant number of developers have stopped looking at code, trusting the bots. I've seen major security regressions, bot rolled auth implementations, and falsified benchmarks... just this last week, all in projects that get hyped.
Rule #2 - do not blindly trust Ai output
It's not stupid if it works.
Concretely, I would like to see it benchmarked:
Have Opus 5.5 implement the authentication, with a 6.1 Sol reviewer. Out of 100 times, how often is the result insecure?
I wouldn't be surprised if it's zero times.
There is also the possibility of specifically prompting for a security audit on top, which would perhaps further lower the risk of overlooked issues.
Now one can say that due to the non-deterministic nature of the tools, there is always a non-zero chance of producing insecure implementation.
That's not really important here - obviously manual review isn't deterministic either, the risk will never be zero.
> Have Opus 5.5 implement the authentication
“Authentication” is extremely far from being the only relevant characteristic of security. That in fact is the whole problem with security: it’s a pervasive concern that has to be considered throughout a system, at every level.
The way the bots fundamentally work means that we can never trust their output. We have ample cases at work where bots catch human mistakes and humans catch bot mistakes. The important aspect here is not completely handing things over to the bots.
> I wouldn't be surprised if it's zero times.
Statements like this concern me.
If you're so concerned you can create such a benchmark and show us all how wrong my guess is.
Instead I mostly read such opinions based on active imagination and no concrete experience with using AI for such work in a setup as described.
7 replies →
> Not only this, a significant number of developers have stopped looking at code
This is what I find frustrating about the whole AI/slop debate. Developers stopped looking at code a long time ago... Well before LLMs. Take for example a typical nodejs project with its 100s of dependencies. Is a developer going to look through the layers upon layers of code running their app? Of course not, there's to much of it. We've seen supply chain attacks even this week.
Actually what I think will happen is that all those abstractions, dependencies will become irrelevant now that an LLM can just write custom code to solve the business problem. This will massively reduce the supply chain problem.
Then have LLMs dedicated to scanning codebases for security issues.