As mentioned elsewhere, the benchmark introduces bad pelicans in the training set. What I'm curious about however, if it's possible for a human artist to "poison" the benchmark by releasing some really good pelicans svgs and have all future models output their version.
If that were the case then it'd do a way better job. Think experienced artist level.
how would great pelicans make their way into the training set?
what they do have are many different pelicans and people helpfully rating them in the comments.
As mentioned elsewhere, the benchmark introduces bad pelicans in the training set. What I'm curious about however, if it's possible for a human artist to "poison" the benchmark by releasing some really good pelicans svgs and have all future models output their version.
That’s covered in the article
[dead]