Comment by lazide

Humans of course will screw at least 1% of the time, at least judged retroactively.

The fun part is, even if you don’t change anything, you’ll likely get a different 1% set of errors each time no matter how perfect your judges.

10% seems pretty high, but it really all depends on what you’re evaluating. If it’s all weird edge cases….