Comment by badsectoracula
10 hours ago
> I know someone is going to say "why not just ask an LLM to simulate a range of users?" The answer is very simple - I want to speak to real people. People are brilliant! They can make you laugh, you can see their cat when it wanders on to the call, they bring a unique perspective to the problem, and they're really happy when you give them a €25 voucher. Some will gladly do it for free and make you happy!
So what the author actually paid for was to interact with humans and the README checking was secondary - because, really, my own first thought was literally to ask an LLM check and try to follow the instructions in the README and pretty much any decent LLM (including several local ones) would be able to check if they're adequate and even suggest improvements (just don't let them write it for you :-P).
Agents and people catch different things. My agent reviewers find real bugs, like pinch-zoom breaking while a finger is on the joystick. But none of them said "this is boring". I did, on my phone, when I walked my character up to a house and nothing happened.
Yes, an agent wont tell you if something is boring or funny, at least not unless you ask (and i don't think its response would have much value) but it will tell you if your instructions can be followed or not - which was the main thing the author wanted to test against, at least as far as i understood.
My point wasn't that you can use agents for all feedback, but for the given case of testing your readme file's instructions (the article's title even mentions it is about the readme file) you can certainly use agents for that.