Comment by modeless

The best world model research I know of today is Dreamer 4: https://danijar.com/project/dreamer4/. Here is an interesting interview with the author: https://www.talkrl.com/episodes/danijar-hafner-on-dreamer-v4

Training on 2,500 hours of prerecorded video of people playing Minecraft, they produce a neural net world model of Minecraft. It is basically a learned Minecraft simulator. You can actually play Minecraft in it, in real time.

They then train a neural net agent to play Minecraft and achieve specific goals all the way up to obtaining diamonds. But the agent never plays the real game of Minecraft during training. It only plays in the world model. The agent is trained in its own imagination. Of course this is why it is called Dreamer.

The advantage of this is that once you have a world model, no extra real data is required to train agents. The only input to the system is a relatively small dataset of prerecorded video of people playing Minecraft, and the output is an agent that can achieve specific goals in the world. Traditionally this would require many orders of magnitude more real data to achieve, and the real data would need to be focused on the specific goals you want the agent to achieve. World models are a great way to cheaply amplify a small amount of undifferentiated real data into a large amount of goal-directed synthetic data.

Now, Minecraft itself is already a world model that is cheap to run, so a learned world model of Minecraft may not seem that useful. Minecraft is just a testbed. World models are very appealing for domains where it is expensive to gather real data, like robotics. I recommend listening to the interview above if you want to know more.

World models can also be useful in and of themselves, as games that you can play, or to generate videos. But I think their most important application will be in training agents.