Comment by michaelbuckbee
2 days ago
Like Simon concludes the article, the main use of this isn't to say which model is "better", but to try and poke at the model to sort out things like quality vs cost vs speed.
So I put together a quick comparison of the last couple iterations of Opus, Fable and now Kimi.
Kimi is cheapest by 5x but also slowest by 2x
Personally I'd consider the three middle ones to be failing, in the typical "Gemini/Google" fashion in that the model is doing more than what the prompt asks. The prompt asks for SVG, yet the model is providing more.
Edit: Actually, looking at the K2.6 response, that's borderline failing too, it's using HTML+CSS+SVG, not just SVG, again failing to follow the prompt properly.
By the way, that website seems like a black hole for information, it says "Expires in 6 days" in the top right which seems really weird for a page hosting couple of KB of data at most.