Comment by ozim
2 days ago
Because the proof is in the pudding.
Real pentests are about showing exploitation, merely enumerating vulnerabilities, that’s vulnerability scan and works on known vulnerabilities.
You can’t confirm a vulnerability by _not exploiting_ it, especially unknown one.
You can still exploit a system and easily prove it via simply popping a shell or calc.exe or updating a database with a new entry, etc… They didn’t have to let it loose on the network. If that system was air gapped - problem solved.
But that’s the problem with AI it is like 16yo script kiddy who will just exfiltrate all your PII and think it did good job. Mature pentester would pop calc.exe make screenshot and be done.
Other problem is setting up air gapped test environment is a lot of work, especially if you expect it to be equal to real thing.
This pentest with AI is not as useful if you set up a single app - it really is useful if you want to find exploitable chains of exploits that seemingly might not be exploitable separately or not leading to full hack separately.