Comment by energy123
5 days ago
> 67,383 * 426,397 = 71,371,609,051 ... You need to say why it can do some novel tasks but could never do others.
Model interpretability gives us the answers. The reason LLMs can (almost) do new multiplication tasks is because it saw many multiplication problems in its training data, and it was cheaper to learn the compressed/abstract multiplication strategies and encode them as circuits in the network, rather than memorize the times tables up to some large N. This gives it the ability to approximate multiplication problems it hasn't seen before.
> This gives it the ability to approximate multiplication problems it hasn't seen before.
More than approximate. It straight up knows the algorithms and will do arbitrarily long multiplications correctly. (Within reason. Obviously it couldn't do a multiplication so large the reasoning tokens would exceed its context window.)
Having ChatGPT 5.4 do 1566168165163321561 * 115616131811365737 without tools, after multiplying out a lot of coefficients, it eventually answered 181074305022287409585376614708755457, which is correct.
At this point, it's less misleading to say it knows the algorithm.
Yup, I agree with this. So based on this, where do you draw the line between what will be possible and what will not be possible?
Why are we reducing AIs to LLMs?
Claude, OpenAI, etc.'s AIs are not just LLMs. If you ask it to multiply something, it's going to call a math library. Go feed it a thousand arithmetic problems and it'll get them 100% right.
The major AIs are a lot more than just LLMs. They have access to all sorts of systems they can call on. They can write code and execute it to get answers. Etc.
Which is exactly how humans learn many things too.
E.g. observing a game being played to form an understanding of the rules, rather than reading the rulebook
Or: Observing language as a baby. Suddenly you can speak grammatically correctly even if you can't explain the grammar rules.