Comment by versteegen

I agree. But I think you're missing that LLMs can internalise a lot of the thinking process in their layers without explicit CoT. That System 1-style reasoning is bounded depth computation but very, very broad. Yudkowsky called it "cached thoughts" and I think it's an incredibly important idea [1]. It's really stiking how the best LLMs don't even need to think where smaller LLMs do.

So as more thinking is cached in their weights through increased RL training, those weights are doing more useful work and the efficiency is increasing.

[1] https://www.lesswrong.com/posts/2MD3NMLBPCqPfnfre/cached-tho...