Comment by oh_no

> Impressive vision benchmarking. If the vision model is truly as good as astra, that would make it best in the world.

Sure, it leads in one unpopular benchmark with internal numbers.

But look at the third benchmark. It only has Sol 5.6 from OpenAI and it shows a 12.8. I don't know this benchmark but AA's GDP.pdf listing has Sol 5.6 Max at a 27, even non-reasoning beats the 12.8. It's extremely weird to cherry pick Sol 5.6 and then also lie about the published third party benchmark score. I'd love AI companies to stop lying about this stuff.

AA - https://artificialanalysis.ai/evaluations/gdp-pdf?models=cla... Mistral announcement - https://mistral.ai/_astro/multimodal-benchmarks---gdp.pdf---...