Comment by QuesnayJr

They thought it because we have an existence proof: gpt-3.5-turbo-instruct can play chess at a decent level.

That was the point of the post (though you have to read it to the end to see this). That one model can play chess pretty well, while the free models and OpenAI's later models can't. That's weird.