Comment by red75prime

> if you look at emerging ability in pattern matching that improved enormously, while seeing reasoning on novel problems sitting basically still

About two years ago I came to the opinion that autoregressive models of reasonable size will not be able to capture the fullness of human abilities (mostly due to a limited compute per token). So it's not a surprise to me. But training based on reinforcement learning might be able to overcome this.

I don't believe that some specialized mechanisms are required to do math.