← Back to context

Comment by simonw

2 months ago

I wrote about that possibility here: https://simonwillison.net/2025/Nov/13/training-for-pelicans-...

Hi Simon! Love your work! Our of curiosity - how many pelican-cycling samples do you produce. Curious about the variance here. Thanks!

Aiden is perhaps misinformed. From a Bing search performed just now.

> Yes, I am familiar with the "pelican riding a bicycle" SVG generation test. It is a benchmark for evaluating the ability of AI models, particularly large language models (LLMs) and multi-modal systems, to generate original, high-quality SVG vector graphics based on a deliberately unusual and complex prompt. The benchmark was popularized by Simon Willison, who selected the prompt because:

  • Web search-based RAG is very different from having something embedded in a model's training data, though.

    • ChatGPT website gives a similar answer. Are they running RAG, or the model?

      > Yes — I’m familiar with the “pelican riding a bicycle” SVG generation test.

      > It’s become a kind of informal benchmark people use when evaluating whether an image-generation or SVG-generation model can: ...

      1 reply →

[flagged]

  • Condescending and disrespectful to whom? Everybody wholsale? This doesnt seem reasonable? Please elaborate.

    • Not sure if I'd use the same descriptions so pointedly, but I can see what they mean.

      It's perfectly fine to link for convenience, but it does feel a little disrespectful/SEO-y to not 'continue the conversation'. A summary in the very least, how exactly it pertains. Sell us.

      In a sense, link-dropping [alone] is saying: "go read this and establish my rhetorical/social position, I'm done here"

      Imagine meeting an author/producer/whatever you liked. You'd want to talk about their work, how they created it, the impact it had, and so on. Now imagine if they did that... or if they waved their hand vaguely at a catalog.

      20 replies →