Comment by lelanthran
4 hours ago
> I would expect code written by the latest frontier models with a "find all security issues make no mistakes" pass to be more secure than average human code written in the pre-AI era (not a high bar).
"If you write code as cleverly as you can, you are, by definition, too stupid to debug it."
Do you disagree with this? Because if you don't, then the rate of bugs/RCEs in a completely AI-written-software world will be exactly the same as the rate of bugs in a completely human-written-software world.
A combination of human-authorship + AI-review, with absolutely no AI-written code is going to have the lowest rate of bugs, because there is always one entity who understands the code (because they wrote it) and one entity who is smarter than the author who can find bugs.
The AI-generated code I see from SOTA models is absolute spaghetti, and they start running into problems spotting bugs after even a small number of refinements/changes to the software (say, 50).
The problem with all this spaghetti is that they are quick to write something that even they can't easily read. Even mediocre but experienced humans are not so eager to write something that they can't easily debug.
IOW, for any non-trivial problem, with AI-generated code you are guaranteed to get spaghetti that it cannot understand. With humans, it's only a probability that that will happen.
Well the AI machines don't have to rely on cleverness to find vulnerabilities, so maybe they're not limited by Kernighan's Law. Does throwing virtually unlimited sufficiently clever monkeys at spaghetti code find these vulnerabilities better than humans? Can you prove they can't?
At the point you starts asking to prove a negative, like "Can you prove that there's no God?", you should have the self-awareness to realise that no argument will convince you, because you are proceeding on the power of belief and belief alone.
If you don't have the self-awareness to realise that you are asking the same question as "Can you prove there's no God?", then I'm afraid no proof will ever be sufficient for you.