Comment by digitalPhonix
5 days ago
I can do it with a scratch pad. And I can also tell you when the calculation exceeds what I can do in my head and when I need a scratch pad. I can also check a long multiplication answer in my head (casting 9s, last digit etc.) and tell if there’s a mistake.
The LLMs also have access to a scratch pad. And importantly don’t know when they need to use it (as in, they will sometimes get long multiplication right if you ask them to show their work but if you don’t ask them to they will almost certainly get it wrong).
> And importantly don’t know when they need to use it
patently false, but hey at least you’re able to see the parallel between you with a scratch pad and an LLM with a python terminal
Sure, lets test that:
https://chatgpt.com/s/t_69c420f3118081919cf525123e39598c
https://chatgpt.com/s/t_69c4215daeb481919fdaf22498fb0c4f
Do you have a different definition of false? I'm referring to their reasoning context as their scratch pad if that wasn't clear.
The context is the scratch pad. LLMs have perfect recall (ignoring "lost in the middle") across the entire context, unlike humans. LLMs "think on paper."