Comment by threethirtytwo

> The caveat is: It depends on the task.

I think the line of criticism around LLMs sucking at chess makes more sense when you understand what the AI companies are saying about the future trajectory of these models.

The entire recursive self improvement story falls apart once you point out that there is not much "cross domain transfer learning". Meaning that training an LLM to become good at coding, math, etc, will eventually transfer into them being good at other skills that were not explicitly trained for.

Using games like chess which have little economic value is actually a good test for this. What's even more surprising about them sucking at chess is how much information about chess strategy exists in the training data.

There’s multiple databases of games in algebraic notation. You can also, very easily rl train on pitting models against one another, even without mcts.

> Are there reams of chess moves that the model can train off of? No.

This is as false as something can possibly be. There are open databases of millions of chess games spanning hundreds of years.

>Are there reams of chess moves that the model can train off of? No.

For real??

[dead]

[flagged]