Comment by bilbo-b-baggins
4 hours ago
Devin shows model thinking.
I’m pretty sure the big bois don’t do it because it would undermine “confidence”.
Seeing a model output “Oh I should just delete blah. Wait blah is a production service, I shouldn’t touch that. Maybe I can gain access to blah? Oh the aws cli isn’t signed in to blah. I see kubectl has access to blah though! Wait, I should ask user permission first.”
Yeaaaaah. Thinking tokens are fuckin’ wild.
Idk I feel like the more likely answer is to prevent distillation. Having the thinking is definitely better UX (oftentimes, I don’t know if Codex is just hanging, which it often does, or working in silence).
Running some models locally and seeing these thinking tokens was quite the experience. I never saw an LLM so "unsure" about virtually everything.
I doubt most users would look at them if they were available. More likely they don’t want to stream distillation material.