← Back to context

Comment by wren6991

8 hours ago

The benchmark is saturated. Frontier models are tested with an armadillo in fishnet tights jaywalking on Mars.

10 comments

wren6991

Reply
simonw  5 hours ago

> The benchmark is saturated. Frontier models are tested with an armadillo in fishnet tights jaywalking on Mars.

OK well I couldn't resist this one:

  llm -m claude-opus-5.5 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars'
  llm -m gpt-6.1-sol 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars'
  llm -m gemini-3.8-flash 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars'
  llm -m mistral/mistral-large-4 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars'

Default reasoning levels for each: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...