Comment by user43928
1 day ago
If you're so concerned you can create such a benchmark and show us all how wrong my guess is.
Instead I mostly read such opinions based on active imagination and no concrete experience with using AI for such work in a setup as described.
I suspect you would disregard or find fault in any benchmark I or others provide you.
Alternatively, if you want to see a benchmark so badly, you can create such a benchmark for your guess. I would imagine you would have more faith in the results of your own work.
If someone were to genuinely tell me that with the two SOTA models with the review step, severe security issues were left in the code, I would believe it.
That's not what anyone tells me though. I only hear theorizing and vague references to how AI failed people.
You're spot on that the thing I trust most is my own experience. I'm mostly working on mobile apps, my project does not have user accounts, so I couldn't tell you about authentication.
However, it is my experience that these models got very good at catching high-priority issues if you use the review step as I described.
Thus I theorize that SOTA models are likely to produce secure implementations.
"secure implementation" is vague, for example, here is an OpenCode issue (links to related PRs/follow ups) that I worked with their team through to improve security.
Our hunch is that the Ai told them this was ok to do, but it broke invariants users relied on. Perhaps it was not "hackable" in the sense I think you are getting at, but it did open a very big security hole through choices and design. The critical thinking was lacking (imo).
https://news.ycombinator.com/item?id=50133401
4 replies →