Comment by JohnKemeny
2 hours ago
Are you saying that modern LLMs cannot play chess now, or that LLMs (GPT architecture) cannot be trained to play chess well?
Or are you saying that neural networks in general cannot (practically) be trained to be an above-average chess player?
Or are you saying that it depends on the input? Would it be better if they were given a picture/drawing/ascii art of the board? If so, surely they can produce it at will?
Neural Nets can be trained to play chess very well and have been doing so for a long time (see Stockfish and Leela as some of the most popular/strongest ones - top GMs have no chance against them), but these are dedicated models, where the game rules are encoded in the learning process, as opposed to large language models which are natural language processing models. Technically you can give an LLM a lot of chess books and games and it will be able to spit out chess notation. Put a webapp on top that renders text moves to the board and it looks like it's playing chess. But it isn't really.
https://en.wikipedia.org/wiki/Chinese_room I think about this once in a while. At some point if it does the thing almost perfectly is it still not doing the thing?
I suppose if you all you need is a good enough opponent for the average person out there, sure this is good enough.
I was more talking in reference to why the LLMs in the above linked paper were producing so many illegal moves, and it is because they are not hard constrained by the rules of the game. Of course, a loop can prompt until a valid move is produced and then rendered on a screen. But why do this? I suppose, who am I to say what should be done or not, but a specialized tool being better than a general one at its specific job isn't particularly surprising.