Do you have a budget per-player of cloud usage? What happens if people really like the game and play it so much it starts getting expensive to keep running? I guess at $0.79 / Mtok llama70B is pretty affordable, but a per-player opex seems hard to handle without a subscription model.
Our initial plan was to simply ask enough for the game that the price would cover the costs on average... but that means that we're basically encouraged to have people play the game as little as possible? We're looking into some kind of subscription now, it sounds weird but I do think it's a better incentive in this case. Plus we can actually ask for less upfront.
Do you think there's a path where you can pregenerate popular paths of dialogue to avoid LLM inference costs for every player? And possibly pair it with a lightweight local LLM to slightly adapt the responses? While still shelling out to a larger model when users go "off the rails"?
Not the founder, but having run conversational agents at decent scale, I don't think the cost actually matters much early on.
It's almost always better to pay more for the smarter model, than to potentially give a worse player experience.
If they had 1M+ players there would certainly be room to optimize, but starting out you'd certainly spend more trying engineer the model switcher than you would save in token costs.
This is fantastic. I think it’s nailed in the substack what was missing from a lot of these LLM driven NPCs that did not feel authentic. I have a couple of follow-up questions on specifics relating to analysis of behaviour with LLMs (in game-dev myself). Would it be possible to speak to you directly on them?
Hey! Robotopia looks awesome, I'm excited to try it out when it launches. How do you convert the LLM output to actions? Is there more broad actions available (ie like creating any object, moving anything anywhere) exposed to the LLM or is it more specific tools it can call?
Thanks :)
It may sound insane but we convert actions to Python functions then ask the LLM to write a python script that actually runs in IronPython inside the game.
Then we have a visual Behavior Tree system to let our designer define the actions. So yeah, they got a bunch of general actions like walk, talk, follow, interact etc.
PS: I think MCP/Tool Calls are a boondoggle and LLMs yearn to just run code. It's crazy how much better this works than JSON schema etc.
Yes, but every language is going to be a "port", not something contracted out like traditional localization.
I haven't decided how exactly but language conversion will land somewhere between these two extremes:
1. (expensive) pick a suite of "native" models (eg. models from China), TTS, ASR. Rewrite all the prompts in the target language. Revalidate all characters by hand
2. (cheap) slap a translation model around input and output and let the game run in English internally. My gut feeling is that this could have very poor results though and increase latency.
It's definitely a research project, this has never been done before.
The "big" one is Llama3.3-70b on the cloud, right now. On GroqCloud in fact, but we have a cloud router that gives us several backups if Groq abandoned us.
We use a ton of smaller models (embeddings, vibe checks, TTS, ASR, etc) and if we had enough scale we'll try to run those locally for users that have big enough GPUs.
(You mean the voxel grid visibility from 2014?! I'm sure I did at the time... but I left MC in 2020 so don't even remember my own algorithm right now)
Another game that has LLM powered NPCs is the f2p action game from China called "Where Winds Meet" and players came up with all sorts of hilarious ways to cheat quests and other fun stuff via prompt injections.
This is an incredible foretaste of what AI can enable in gaming. Not replacing humans (the creators here are former leaders from Minecraft), but rather simply unlocking more fun gameplay by offering creativity, humor, and branched storytelling customized to the player.
Hah from my knowledge of traditional AAA, there is 0 chance any AAA in development right now uses LLMs. A lot of them don't even use it for coding and gamedevs' mood about AI is abysmal.
What's interesting is you might not want to see de novo AI-generated storytelling (slop factor), but you might really like the way AI can make a story crafted by humans more interactive.
We want to gamify prompt hacking and give people an UI to add/remove chunks of the system prompt. It'll be unlocked by collecting widgets around the place.
It's a stylistic choice for sure. A little better than that is straight in uncanny valley, and human-level is too high latency and too expensive for us.
We found that this level of crappy works great, in practice, plus it runs on-device! We use Rhasspy Piper to generate them.
I would personally avoid voices that skew too close to common tiktok TTS ai. Currently the heavy robots with the lower bassier voices sell that clunky robot voice vibe much better, but some of the more generic voices immediately take me out.
Hey, Tommaso here, I'm one of the founders of the Robotopia studio. I didn't expect to see this here! Ask me anything :)
Do you have a budget per-player of cloud usage? What happens if people really like the game and play it so much it starts getting expensive to keep running? I guess at $0.79 / Mtok llama70B is pretty affordable, but a per-player opex seems hard to handle without a subscription model.
Our initial plan was to simply ask enough for the game that the price would cover the costs on average... but that means that we're basically encouraged to have people play the game as little as possible? We're looking into some kind of subscription now, it sounds weird but I do think it's a better incentive in this case. Plus we can actually ask for less upfront.
Do you think there's a path where you can pregenerate popular paths of dialogue to avoid LLM inference costs for every player? And possibly pair it with a lightweight local LLM to slightly adapt the responses? While still shelling out to a larger model when users go "off the rails"?
Not the founder, but having run conversational agents at decent scale, I don't think the cost actually matters much early on.
It's almost always better to pay more for the smarter model, than to potentially give a worse player experience.
If they had 1M+ players there would certainly be room to optimize, but starting out you'd certainly spend more trying engineer the model switcher than you would save in token costs.
1 reply →
This is fantastic. I think it’s nailed in the substack what was missing from a lot of these LLM driven NPCs that did not feel authentic. I have a couple of follow-up questions on specifics relating to analysis of behaviour with LLMs (in game-dev myself). Would it be possible to speak to you directly on them?
Thanks :) If you want I'm on the discord linked on our landing page, it's fun stuff to talk about!
1 reply →
Hey! Robotopia looks awesome, I'm excited to try it out when it launches. How do you convert the LLM output to actions? Is there more broad actions available (ie like creating any object, moving anything anywhere) exposed to the LLM or is it more specific tools it can call?
Thanks :) It may sound insane but we convert actions to Python functions then ask the LLM to write a python script that actually runs in IronPython inside the game. Then we have a visual Behavior Tree system to let our designer define the actions. So yeah, they got a bunch of general actions like walk, talk, follow, interact etc.
PS: I think MCP/Tool Calls are a boondoggle and LLMs yearn to just run code. It's crazy how much better this works than JSON schema etc.
3 replies →
This has insanely incredible potential for language learning. Do you plan to implement support for additional languages?
Yes, but every language is going to be a "port", not something contracted out like traditional localization. I haven't decided how exactly but language conversion will land somewhere between these two extremes: 1. (expensive) pick a suite of "native" models (eg. models from China), TTS, ASR. Rewrite all the prompts in the target language. Revalidate all characters by hand 2. (cheap) slap a translation model around input and output and let the game run in English internally. My gut feeling is that this could have very poor results though and increase latency.
It's definitely a research project, this has never been done before.
Are the LLMs run on-device, or does this use cloud compute?
(Off-topic AMA question: Did you see my voxel grid visibility post?)
The "big" one is Llama3.3-70b on the cloud, right now. On GroqCloud in fact, but we have a cloud router that gives us several backups if Groq abandoned us.
We use a ton of smaller models (embeddings, vibe checks, TTS, ASR, etc) and if we had enough scale we'll try to run those locally for users that have big enough GPUs.
(You mean the voxel grid visibility from 2014?! I'm sure I did at the time... but I left MC in 2020 so don't even remember my own algorithm right now)
8 replies →
Another game that has LLM powered NPCs is the f2p action game from China called "Where Winds Meet" and players came up with all sorts of hilarious ways to cheat quests and other fun stuff via prompt injections.
https://www.dexerto.com/gaming/where-winds-meet-players-are-...
https://www.rockpapershotgun.com/where-winds-meet-player-con...
I had no idea this game had LLM NPCs. Interesting
This is an incredible foretaste of what AI can enable in gaming. Not replacing humans (the creators here are former leaders from Minecraft), but rather simply unlocking more fun gameplay by offering creativity, humor, and branched storytelling customized to the player.
I strongly suspect that the advent of LLMs stalled the new elder scrolls game another 5-6 years.
Hah from my knowledge of traditional AAA, there is 0 chance any AAA in development right now uses LLMs. A lot of them don't even use it for coding and gamedevs' mood about AI is abysmal.
2 replies →
What's interesting is you might not want to see de novo AI-generated storytelling (slop factor), but you might really like the way AI can make a story crafted by humans more interactive.
1 reply →
Why? Because they feel like it needs to be a part of the game?
I'm imagining a version of this where you have to use various prompt- or data-centric attacks to navigate scenarios
We want to gamify prompt hacking and give people an UI to add/remove chunks of the system prompt. It'll be unlocked by collecting widgets around the place.
This looks like a lot of fun. Is there a way to use text rather than speech for input? I'm not particularly fond of my voice getting sent to an LLM.
Yeah, there's a toggle to type you can switch at any time, it actually lowers latency.
I like the concept. Though, they couldn't have found better text-to-speech voices? Or is it meant to be humorous how bad they are.
It's a stylistic choice for sure. A little better than that is straight in uncanny valley, and human-level is too high latency and too expensive for us. We found that this level of crappy works great, in practice, plus it runs on-device! We use Rhasspy Piper to generate them.
I would personally avoid voices that skew too close to common tiktok TTS ai. Currently the heavy robots with the lower bassier voices sell that clunky robot voice vibe much better, but some of the more generic voices immediately take me out.
1 reply →
I’m so excited to see LLMs used more creatively in video games. So many new mechanics can be unlocked with LLMs as judges
Agreed!
Some other cool ones I've seen: https://store.steampowered.com/app/2542850/1001_Nights/ https://www.playsuckup.com/
Robotopia was very inspired by suck up. First LLM game that kinda cracked the 3d world
Max Headroom?
Nice concept and good try!
Thanks :)