Comment by eru

Computers managed to beat the world's best humans at chess long before they managed how learn how to walk. What's easy and hard for humans isn't necessarily a good guide to what's easy or hard for a computer.

(And I'm fairly sure it would be pretty easy to build a system that uses an LLM and a few other small components to beat Pokemon Red. The experiment you are talking about is deliberately hobbled by using a stock LLM without any such tools to make the whole thing entertaining. But when you are trading, you'd want to give your LLM as much help as possible.)