Comment by hannasanarion

Probably not a clear divide, but it might be some kind of divide, and it might be the opposite of intuition.

The big advantage the LLMs have is their huge context windows, ability to parallelize and delegate, and no attention fatigue or need to sleep. They can think about random combinations of functions and modules all day long until they find a combination that's exploitable. That's hard for a person to do, and its the kind of exploit that probably would never be found otherwise except by another llm-powered thing. Not being a security expert myself, I gather that these AI vulns tend to be small needles in very large haystacks that normal people just would never have the patience to search.

But if you found something yourself, then it's something that's simple enough and involves a small enough number of moving parts that you can keep it in your head all at once and recognize the problem and correctly reason about its consequneces, which probably means it's easier to exploit, and therefore more dangerous if made public.

It feels a little backwards, and it might still be backwards, but if there's any benefit i think it's along those lines.