Comment by himata4113

This seems to be a hit and a miss, there's no visual indication that opus 5.5 is better even though in 9 tests out of 10 it will outperform astra especially when it comes to niche domains and will absolutely destroy them in anything requiring raw intelligence rather than a naive search, naive search is what makes openai models extremely capable in scientific fields though.

The score hit on smaller models is a welcome change especially when compared to models that nearly 10 times larger.

Grok being punished quite a lot is also a very welcome change as the model is frankly not that good.

I would love to see few more points knocked off haiku by introducing steerability/creativity benchmarks.