Comment by chefandy
3 months ago
TBH I haven't seen a single use of LLMs in games that wasn't better served by traditional algorithms beyond less repetitive NPC interactions. Maybe once they get good enough to create usable rigged and textured meshes with enough control to work in-game? They can't create a story on the fly that's reliable enough to be a compelling accompaniment to a coherent game plot. Maps and such don't seem to need anything beyond what current procedural algorithms provide, and they're still working with premade assets— the implementations I've seen can't even reliably place static meshes on the ground in believable positions. And as far as NPCs go— how far does that actually go? It's pure novelty worth far less than an hour of time. Let's even say you get a guided plot progression worded on the fly using an LLM, is that even as good, let alone better, than a dialog tree put together by a professional writer?
This Civ idea at least seems like a new approach to some extent, but it still seems to conceptually not add much. Even if not, learning that it doesn't it's still worthwhile. But almost universally these ideas seem to be either buzzwordy solutions in search of problems, or a cheaper-than-people source of creativity with some serious quality tradeoffs and still require far too much developer wrangling to actually save money.
I'm a tech artist so I'm a bit biased towards the value of human creativity, but also likely the primary demographic for LLM tools in game dev. I am, so far, not compelled.
It's been posted in-depth a few times across this forum to varying degrees by game developers - I was initially very excited about the implementation of LLM's in NPC interactions, until I read some of these posts. The gist of it was - the thing that makes a game fundamentally a game is its constraints. LLM-based NPC's fundamentally break these constraints in a way that is not testable or predictable by the developer and will inevitably destroy the gameplay experience (at least with current technology).
Yeah, same. Epic's Matrix demo implemented it and even without a plot, the interactions were so heavily guided that the distinction was pointless. So you can find out what that NPCs spous's name is and their favorite color. It's that neat? Sure it's neat. It's it going to make it a better game? Probably less than hiring another good writer to make NPC dialog. To be truly useful, I think they would have to be able to affect the world in meaningful ways that worked with the game plot, and again, when you clamp that down as much as you'd need to to still have a plot, you're looking at a fancy decision tree.
Nobody will know for sure until a big budget game is actually released with a serious effort behind its NPCs.
I can't see anything that Gen AI NPCs would add unless maybe you're talking about a Sims kind of game where the interactions are the point, and they don't have to adhere to a defined progression. Other than that, it's a chat bot. We already have chatbots and having them in the context of a video game doesn't seem like it would add anything revolutionary to that product. And would that fundamentally stand a chance of being as compelling to socially-focused role-playing gamers as online games?
This is my field so I'm always looking for the angle that new tech will take. I still rank this lower than VR— with all of its problems— for potential to significantly change player interactions. Tooling to make games is a different story, but for actual use in games? I don't see it yet.
Sandbox games are probably where they will shine. Imagine being able to play Minecraft, and tell a prompt to generate a world that resembles Tatooine, or a vampire-themed mansion. Expectations are lower with sandbox games, so there's no risk of breaking immersion like would happen with an LLM Elder Scrolls game when someone tricks in NPC into solving problems in python.
Granted, I'm certain there will be copyrights issues associated with this capability, which is why I don't think it will be established game companies who first take a crack at this approach.
8 replies →
As someone who has tried a lot of role-play models, I think there is definitely value in what LLMs (or similar tech) can add to NPCs, it's just most people don't know how to prompt for it.
Using the RP models, over time I've found certain things that can guide them to creating better stories; an agent system is much easier to use but even using single character cards it's not hard to stuff them with a narrator and several individual characters in one go. I recently switched from kunoichi (8b, decent) to an Aria derivative (13b, much better).
In the majority of role-play stories I do now, it's super easy to refine the prompt so that characters don't necessarily provide pointless details + avoid all the common tropes, especially with newer models.
Maybe I should make a PoC, would be a fun project. But yeah I agree that chatting to an NPC about its day doesn't necessarily make for great gameplay - but it's relatively easy now to guide it into interesting scenarios/experiences, which _does_ make for great gameplay.
Ie the wife of the hunter you murdered in a fantasy game; normally we just think that we killed a character in a game - but when the hunter's wife decides in the background to train with a sword so that she can avenge her husband, then finally comes to find you and calls you out for murdering her husband - suddenly it's murder, and a revenge story. It's not too hard to prevent a decent model from injecting fluff (like where she bought her sword and how much for) into it.
Edit: just tested this to see what would happen; I first walked into a cottage, grandfather and his young granddaughter, stabbed him in front of her and ran away (spent the next 2 years of "game time" in a forest hiding away). Character motivations updates for the granddaughter were essentially: distraught, vowing revenge, travelling around to hone her skills, speaking with unsavoury types in taverns to find my whereabouts, finding & confronting me, killing me. I was able to query it for "3 dialogue options/actions with percent chances and distinct outcomes in JSON format" which it gave, the chance of her forgiving me was 0.01% which I suppose is fair enough. It did fail to create nice JSON tho, the model is not fine-tuned for that at all.
But it's definitely possibly with multiple loras/prompts/queries to extract dynamic dialogue options, actions, stats, percent chances for plot/story paths etc. LLMs in games definitely need to be managed by a traditional rules based framework, LLM should only be used for the creative bits. Stats/player skill will always determine who wins a fight, but the fight starting because of dialogue or past events could totally be LLM driven.
1 reply →