Comment by agajews
5 hours ago
Hey everyone! I'm Alex, one of the founders of Pantograph. We've spent the last six months building a pretty smart Minecraft model, coming soon to a server near you!
We trained it on about 500k hours of Minecraft videos, and it learned how to fight creepers, build walls and other structures, and explore to find visual goals.
We're considering putting up a public API for larger models like this one, let us know if you'd like to be able to put Pan in your own server :)
What's most interesting about the model isn't the performance that it gets in Minecraft, but how general the method is. When we scale it up, it should be able to act in any kind of video game, as well as robots in the real world (which are really just another video game).
Hi Alex! I'm Ryan, I run a Minecraft theme park recreation server. While our gamemode is very far from vanilla Minecraft (it plays more like a MMORPG), I've been interested in standing up a model like this to help automate agentic testing of new software features in my monolith Bungee/Paper codebase.
In its current state, do you think it can handle non-vanilla Minecraft tasks like interacting with custom UI's, running commands, or doing other unusual tasks in the world?
Probably not the current model, but one of the benefits of doing internet-scale pretraining is that the model has seen a lot of mods already! We think the bigger models will be able to handle some more custom Minecraft worlds.
I would love to be able to add one of these bots to my Minecraft server! I’ve got a family server and we’ve been talking about how to get an AI in there for a while. I tried out some open source harnesses to allow a generic LLM to join, but none of theme were particularly good.
Awesome! We should be a lot better than ordinary LLMs, especially at tasks that require making a lot of decisions in real-time.