Comment by pilaf
4 hours ago
I think it's curious that it has so many shared elements with the latest Astra pelicans: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
- Sun on top right
- Cloud on top left
- Three "speed lines"
- Two feathers on top of the head
- Eye rendered as a black circle with smaller white circle inside
I wonder if the pelican benchmark is converging across models due to past results being used in training.
These are pretty much what a human would draw. Sun rises from the east. A cloud makes the background “sky”. Three lines is the minimum to interpret as movement. Two feathers is standard on every cartoon and illustration.
This comes up every thread. I think we've all noticed how similar they are becoming.
I would guess that they've definitely been trained on previous results, as they obviously share way too many traits at this point to be totally random. That said, I don't think we're seeing any signs of pelicanmaxxing yet from the providers, so it's still a useful (or at least fun) benchmark.
Once all the models produce pristine, elaborate pelicans riding perfectly drawn bicycles, then it'll be time to move on to pigs driving a racecar or something.