Comment by ping00
4 days ago
as a pentester at a Fortune 500: I think you're on the mark with this assessment. Most of our findings (internally) are "best practices"-tier stuff (make sure to use TLS 1.2, cloud config findings from Wiz, occasionally the odd IDOR vuln in an API set, etc.) -- in a purely timeboxed scenario, I'd feel much more confident in an agent's ability to look at a complex system and identify all the 'best practices' kind of stuff vs a human being.
Security teams are expensive and deal with huge streams of data and events on the blue side: seems like human-in-the-loop AI systems are going to be much more effective, especially with the reasoning advances we've seen over the past year or so.
We will have the age of the centaur across all white collar domains. How long that age lasts I don't think is all that relevant before it has even happened.
The question is not human in the loop but how many humans in the loop?
Then I think about what does a team of 3-4 centaurs look like? For me, it looks like the unemployment line. I am sure there are people on this board who are in the top 5% of whatever the domain is in question. They will be part of the centaur while most people are just redundant.
If you try to counter this with a nineteenth century economic heuristic about coal use , I don't think it works.
Every conversation I've been a party to has been premised on humans in the loop; I think fully-automated luxury space vulnerability research is something that only exists in message board imaginations.