Comment by robryan
17 hours ago
GLM 5.3 flash seems to get more excited the longer it has been trying to hunt down a problem. Complete with caps, many exclamation marks and emoji.
It is funny sometimes because the actual issue it traced down was mostly inconsequential.
OMG I think I found a way to center a div!!!
^-- me, on at least 5 separate occasions spanning multiple years.
isn't this just context shifting?
if one were to remove the expressions of excitement from the previous messages would it the model continue to demonstrate that same excitement scaling?
Yes this sounds just like the effect where people new to coding AI negatively berate it like a person and it continues to get worse and make more mistakes because that’s what those tokens are related to.
100%, back when it was Ox Alpha I had a little fun trying to guess what it might be by looking at the reasoning and I consistently laughed at how excited it got
I counted something like 30 different instances of run-on exclamation marks ("!!!!!!!!!!!") and weird mannerisms ("Waitwaitwaitwait.") in just one GLM 5.3 Flash session. Our token budgets are getting eaten up by this stuff...
I expect it's actually not wasted and there's meaning behind what seems like nonsense to us in helping it achieve it's goal. Which is mildly chilling but not unexpected.
I think this is a known phenomenon: even in non-reasoning models, adding useless/filler tokens before an answer improves task performance. The model is doing some computation during the filler. See: https://arxiv.org/html/2404.15758v1
1 reply →