Comment by simonw

You and a few other people, but enough people still appreciate the bit that I'm going to keep doing it.

They're easy enough to skip - click the little "-" icon and you'll collapse the entire sub-thread.

But how else am I supposed to know when we've reached AGI, until I see an absolutely flawless pelican?

All of the pelicans so far have had really weird flaws / quirks so I am always a little interested to see how well these models perform at this task, since I've seen all the past pelicans and have some anchoring.

Seeing a truly flawless pelican would tell me that the model has true visual reasoning capabilities as well as good taste.

I think you're underweighting the Pelican test.

Not only does it give you a super easy-to-grok understanding of the model quality just by looking at the image, but when you compare tokens and costs (both input and output), you really get a good, simple COST x QUALITY evaluation across models.

Simon explains it well: https://simonwillison.net/2026/Jul/16/kimi-k3/#what-can-we-l...

Simon, you should put up a summary table page that you update after every release.

I feel the same way. It was fun at first but has gotten tiresome. Does anyone actually use these models to generate SVGs?

At this point, it's kind of a hackernews thing. Simon posts them as a single comment in the relevant thread. It's okay for this place to have a little bit of a sense of community, and you can just ignore the comment.

Its a nice benchmark. Like hearing the ice cream truck on a summer day.

Yeah. It's something I can do myself in a couple seconds if I want, also on more varied SVG scenes. If this is going to be a benchmark people turn to I'd like to see more effort put into it than just a one-sentence prompt.

Here is a different opinion. I’m always looking forward to see the pelican whenever a new model is released. It’s plain simple to understand, memorable, subtle enough in terms of details, and my favorite part is that you have been doing them consistently long enough for it to be useful for comparing almost everything with anything.

Disagree. They're a nice tradition, but besides that, they're a useful way of eyeballing improvements. I realise labs are likely to be training for Pelicans - but if they're all training for them, the differences in the results are as indicative as they were before labs trained for them.

The 3.6 Flash pelican is just about the best I've seen.

My 2 cents: you don’t have to look at the pelican if you don’t want to.

All the models do this well. It's a test that tell us nothing at this point.

Hard disagree. I love a little bit of whimsy (which I feel the world is lacking more and more everyday) from Simon everytime a new model is announced.

Perhaps freshen it up and extend the test by feeding the model the rendered output so it can iterate once. Assuming a multi-modal model.

I'm happy for Simon to post what he wants, when he wants. He's earned it.

I love them. Keep 'em coming.

Vibe code an extension that autocollapses any post mentioning pelicans and by simonw?

Do something instead of complain

Your comment reads very pedantic with a hint of jealousy. The pelican and xbox controllers are great ways to see how well it can follow direction dealing with svg a difficult format for LLMs to use and testing their spatial vision awareness.

It's just how he is. Prior to LLMs he was cramming a datasette link into every thread. Downvote and move on.

I find Simon's work informative and entertaining; the last thing he can be accused of is low effort. The Pelicans are just a bit of fun icing on top.

I'll split the difference. When it's a blog post there's usually an interesting observation or two, but if it's totally automated? Maybe just do the ones with a post.

I like seeing the pelicans, it's a tradition.

Yeah and it's surely in the training data by now. Long past time to stop.

[flagged]

A new model arrives. The pelican, with uncanny commercial instinct, is never far behind.

Sponsored blogs and paid newsletters are after all, notoriously poor at subsisting on silence :)