Comment by foobiekr

> would never be able to perform well on novel coding or mathematics tasks. We were wrong

I'm not clear at all we were wrong. A lot of the mathematics announcements have been rolled back and "novel coding" is exactly where the LLMs seem to fail on a daily basis - things that are genuinely not represented in the training set.