Comment by sobellian
5 hours ago
A trend I've noticed in results from AI search (not just LLMs) is that they often look obvious or hard to miss in retrospect. But finding them by oneself is more difficult. I personally experience this when looking at engine lines in chess or go. I have also noticed this description in AI-generated proofs or counterexamples to certain theorems. So while we can say, yes, it found public endpoints or poorly configured software or [etc]; sure, but could you have found those? And in what amount of time?
Treasure maps are easy to follow
The scrappy adventurer traverses difficult terrain
and the well capitalized militia group always flies in too, with ease
To put this into game theory lingo, I think this is because the “branching factor” for any kind of research or exploit is extremely large. So looking backwards it doesn’t seem complicated, but looking forwards there are an enormous number of possible next actions.
Similar to finding a filament for lightbulbs, it might seem obvious to use tungsten, but at the time it wasn’t and Edison searched thousands of materials.