Comment by lesspassiveobse
6 hours ago
Switching the thinking from sampled coherent language tokens to raw logits not scored to correspond to any language.
6 hours ago
Switching the thinking from sampled coherent language tokens to raw logits not scored to correspond to any language.
In the research literature this is generally called coconut after the first paper published on the idea. It has advantages and disadvantages. I agree it’s more efficient. But the lack of observability and transferability are real downsides that block its adoption.
the point is it may be a feature. most providers don't want clients reading those and almost nobody provide thinking text anymore.