Comment by simonw
16 hours ago
Pelicans are solid: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Here's Kimi K3 for comparison, though I didn't try that on different reasoning levels: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Update: I ran all three reasoning levels against K3 again: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
I think I have yet to see a pelican where the left leg and pedal are drawn on the left side of the bike.
There have been a few that get that right - most notable Qwen 3.8 27B running on my laptop: https://simonwillison.net/2026/Aug/16/qwen-38-27b/#the-defau...
Also the recent Geminis - https://simonwillison.net/2026/Oct/13/llm-gemini/ - and GPT-6 Astra on Max: https://static.simonwillison.net/static/2026/gpt-pelicans-gr... - and Claude Fable 5.1 https://static.simonwillison.net/static/2026/claude-pelicans...
GLM 5.3 not only got that part right, but animated it: https://crimson-jeri-74.tiiny.site/
To be fair I have a feeling that if you added 'animated' to the SVG request, the output quality from many other models would improve, just because of the extra chain-of-thought work needed to fulfill the more complex request. IMHO the animation requirement should be added to the canonical pelican prompt.
Simon's comment apparently got flagkilled; I vouched for it.
If enough people are flagging it, we may be at a point where the pelican benchmark is truly overdone.
Please do not decrease the useful signal with this noise
Is that even a debate at this point? Doing my job as a netizen to maintain signal/noise ratio
definitely not frontier though, as even on max both legs are in front of the frame; although I am wondering how much getting that right correlates with model intelligence.
The biggest (of many) failing of the pelican test is that it's only one prompt - what matters most these days is how well a model can drive a lengthy agentic tool calling session.
I like the low effort one most.
Can I just say that I love how you are also just always here the moment something drops lmao. Hope you're all well! The pelicans indeed are solid.