Comment by bel8
6 days ago
That's possibly the worst pelican I saw from all recent LLMs.
Meanwhile GLM 5.2 drew a cool self-contained fully animated SVG pelican:
6 days ago
That's possibly the worst pelican I saw from all recent LLMs.
Meanwhile GLM 5.2 drew a cool self-contained fully animated SVG pelican:
Yeah, GLM have been beating Anthropic on the pelicans for a while now.
(I suspect that's more of an indication that Anthropic have chosen not to waste resources training on animals riding vehicles, personally.)
This is interesting, I haven’t actually heard you suggest that the labs are focusing on this benchmark before. Have you come around to this position as a result of the quality of pelicans you’ve been getting?
The reason I thought this was an interesting benchmark is because it’s a non-image generating model creating an image using SVG code, so it kinda spans capabilities.
If an AI lab trained a model specifically for animals riding bicycles it seem trivial to modify the prompt and determine if it was trained specifically for that or if it’s generalized a skill and can also generate a proper orangutan walking on stilts or an armadillo on a skateboard, this sort of thing?
That's what I've been doing - trying different animals in different vehicles. I'd love to find a lab who does a good pelican on a bicycle but sucks at other combinations, but sadly that's not happened yet.
Google Gemini have openly boasted about their animals on vehicles results! https://x.com/JeffDean/status/2024525132266688757
That's one possibility albeit quite charitable. I'd be inclined to think the same personally if GLM 5.2 wasn't also rocking in other areas too.
Just need the legs to interact with the pedals now :D
That's something that Sonnet 5 couldn't get right even on a static SVG.