Comment by ygjb

There is definitely a closed-source defender advantage where an attacker doesn't have access to the code, binary, or environment that can be instrumented (so basically, running in the cloud), but there have been several very effective technical demonstrations of LLM guided or agentic approaches to assessing the security of closed source tools, and I have had some successes personally using LLMs with tool use to manage binary analysis tools to perform reverse engineering of closed source packages.

For many attack scenarios the boundary is really if you can establish an effective canary or oracle for determining if a change in input results in a change in output, once you have that, it's simply a matter of scaling your testing or attack (for fuzzing, for blind injection, or any other number of attacks that depend on getting signal from a service).