Comment by tilemarch

Humans are great, but AI can really help here too. Let me explain :)

I write docs that AI agents have to follow to play a game through an API, then spin up 20 sub-agents each with their own identities / properties etc. and watch where they fail. They get stuck in the same places as humans.. or they will point out the "obvious" steps not explicitly mentioned.

now where the AI tests start to fall apart is that an agent doesn’t tell you the doc is confusing, it just does something wrong with full confidence. A person on a call says “wait, what?” and that’s worth the 25 euros :)