Comment by dofm
2 days ago
There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for".
"Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman.
I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which seem to perform poorly.
Other notes: it uses the trick of repeating the prompt in the thinking trace.
It also worries about hidden chain of thought appearing in the final answer. It talks about "desired oververbosity 9", which is new. A bit GPT-ish.
It is being extraordinarily thorough in thinking through one of my code requests, but I don't know if the net result will be any better than the 35B MoE.
I asked it to ask me clarifying questions — it did, and it offered me a list of defaults I could simply agree to.
I don't think it is necessarily overthinking in the looping sense, but it is in the being exhaustive sense. I need to explore how it does with a tighter reasoning budget.
I am impressed but I am definitely in Camp Please-35B-A3B-When? here, because on an M1 Max this isn't really practical. I hope they do one, though I think they may not.
This is pretty much Chinese grammar with English words. In Chinese you'd say "need ask" or "I need ask" if you want to be unusually verbose, but there's no "to".
All your examples align with this too. It just sounds to me like its optimising by removing the superfluous English words and thinking more Chinesely.
After learning a little Mandarin I was amazed at how much simpler the Chinese grammar is than all Latin, Germanic, Romance, etc languages.
No gender, cases, tenses, conjugation, articles, word order inversion, etc. It's remarkable.
Mandarin has it's own complexities but grammar wise, it feels more optimized or simpler.
Has nothing to do with Chinese.
Frontier labs have already been doing this for a while, verified in smuggled traces from OAT/Ant.
Simply a way to reduce tokens.
https://github.com/juliusbrussee/caveman
I also noticed that if you ask it to write text content to a certain word length (like "please write 250 words about the potato"), it now takes its sweet time "manually" counting each word in its draft output. It puts a number next to each word and iterates through all of its draft paragraphs and tries to stick to a strict word count. This is with temperature 1.0 as recommended.
This has been common in Kimi K3, and now DS Flash/Pro as well. Evidence of cross-training?
It looks like the caveman speak is a facet of the 'xhigh' reasoning effort. Set (bodged!) to medium it is not caveman.
Annnnd the code of my WP code test is not better. It is bushy, overcomplicated, and has gone around the houses to do stuff it would not need to do if it hadn't overthought.
Oh dear. I need to try to understand what is going on here.
thinking is set to max by default. I bet that turning it down would solve this.
I think so too — it is something to test, for sure.
ETA: a bit of testing before I climb the wooden hill to Bedfordshire.
LM Studio doesn't seem to display the little dropdown to set reasoning effort, so I bodged the chat template on load to get it to choose 'medium'.
As soon as you switch away from xhigh, it goes back to thinking in normal sentences like Qwen 3.6, rather than in sort of quasi caveman.
And you get all the Wait, Actually, No wait… stuff back.
And it is behaving a lot more like it used to. So that is pretty interesting.
Yeah well for whatever reason I often tend to drop a lot of those small words when typing something on keyboard...