Comment by verdverm
19 hours ago
> Are you sure that such accuracy for auditing is impossible, and will remain impossible in the future?
No one can be sure either way, but I have high confidence perfectly secure code will remain out of the reach for man and/or machine, especially while they remain based on the transformer architecture.
The benchmark is hardly perfectly secure code.
Was not your concern that the generated code would be significantly less secure than what an average engineer produces?
The goal posts here basically moved toward infinity.
> Was not your concern that the generated code would be significantly less secure than what an average engineer produces?
No, generally I think it possible to improve code quality with agents. The main problem I see to this is eyes free coding and compounding errors therein. With regards to major security issues, I do not think transformers can reliably find them, even when stacked, looped, or teamed. I've seen this setup still miss things or make mistakes.