Comment by HarHarVeryFunny

I'm not sure how you are imagining an LLM "self evaluating". They are feed-forward auto-regressive predictors. The only source for the next word they spit out is what came before and the patterns that contains.

If you ask an LLM to "win like Magnus" it can't because, as above, it is a feed-forward auto-regressive predictor. Do you realize how much different this is to how something like Stockfish (that can play at, and far beyond, Magnus' level) works?

A big LLM is still an LLM - the architecture and operation doesn't change. It just has more capacity to learn, both to learn deep predictive patterns and to memorize factual knowledge.

Does it not occur to you that your beliefs of what LLMs are capable of (generating amazing prose) is totally at odds with what we see them doing? Where is all the amazing prose?