Comment by TeMPOraL

> public misconception about how current AI works

Another case where allowing oneself to anthropomorphise LLMs a little yields better intuitions.

People don't get more skilled by raw reasoning, either. They improve their skills with better textbooks (pre-training data), more practice exercises (RL data), and generally goofing around (self-play).