Comment by emp17344
3 days ago
Emergence within LLMs is pretty much unproven, and this seems like pretty strong evidence that nothing is really “emerging” here.
3 days ago
Emergence within LLMs is pretty much unproven, and this seems like pretty strong evidence that nothing is really “emerging” here.
"Unproven" is in the eye of the beholder, but to me it certainly looks like the sudden take-off in arithmetic ability (to name one example) at approx 10^22 training FLOPs across all LLMs is emergent. (ref. https://hai.stanford.edu/news/examining-emergent-abilities-l...)