Comment by tomrod

12 hours ago

Its a nice benchmark. Like hearing the ice cream truck on a summer day.

It's both.

I agree to rednb that at this point it feels like rather obvious brand building, but also, I agree with you that some value is in it.

It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.

  • Sorry mate, but you sound jealous in all these replies that the Pelican domain isn't your gig. The below is as labored as nitpicks ever get:

    > It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.

    I hope SimonW keeps them coming.

At this point it does not show anything as models are fine tuned on all kinds of benchmarks.

  • I thought the Gemini 3.5 Flash Lite response was quite telling myself. I personally like the Pelican SVG test, to me it is still a charming snapshot of model performance anecdata. No one would argue it's rigorous but I don't think it was ever intended to be.

    I get people burning out on the pelican SVG test alongside the rest of the AI burnout, but I guess for myself I'm just choosing to keep enjoying it while I still can.