Comment by keeda

This is an intriguing observation! LLMs were famously bad at spatial reasoning, until Astra which apparently has a huge improvement. I wonder if that has any bearing on the recent jump in Mathematical performance?

One thing that struck me from Dario's last podcast with Dwarkesh was that he said training LLMs on a diverse set of tasks does not make them better just at those tasks, but they get better at unrelated and other tasks overall. What you described could be a concrete example of how that dynamic works!