Comment by wren6991
16 hours ago
I think this is a known phenomenon: even in non-reasoning models, adding useless/filler tokens before an answer improves task performance. The model is doing some computation during the filler. See: https://arxiv.org/html/2404.15758v1
It is. Processing tokens is the only time model has to do computation, and if you ask it a tough problem, there is some minimal amount of computation it needs to perform to process and solve it - pre-CoT in particular you could guarantee failure by forcing model to be concise, and thus giving it less computational budget than necessary to compute the answer.
(This is I think where people parroting out "stochastic parrot" are stuck even today - not realizing that "predicting next tokens" is hiding arbitrary computation underneath, with token stream acting as input and clock signal...)