Comment by anon373839

I think the idea of emergent properties in LLMs is less and less relevant. Yes, the first few times they scaled pretraining and such models could do few-shot tasks was quite interesting. But the models we use now are really the product of extensive RL hillclimbing on a wide array of curated tasks. What’s interesting now is that the tasks they’ve been fitted to (long running tool call chains interleaved with reasoning tokens) do very useful things.