Comment by beloch
24 days ago
1. Imagine a video game like Red Dead Redemption where each NPC is voiced by AI and can respond to you in a convincingly human fashion. Their responses and even the plot of the whole game can change based on your interactions with NPC's.
2. Imagine a world in which humans can still write books and interactive experiences and find audiences sufficient to earn a living at it.
I really want these two things to be compatible, but I'm not convinced they are. #1 is a gamer's dream, but it's a nightmare for our humanity if it comes at the cost of #2. That's why I'm highly ambivalent about this contest and its results.
> 1. Imagine a video game like Red Dead Redemption where each NPC is voiced by AI and can respond to you in a convincingly human fashion. Their responses and even the plot of the whole game can change based on your interactions with NPC's.
Have you ever gone exploring in Minecraft, or No Man's Sky? Those games are effectively infinite, but I find they run out of interesting generated content after maybe 10 or 20 hours.
The problem is, once you see the outlines of the world generation, your brain kind of fills in the space between. I've seen blue grass, and I've seen purple oceans, so blue grass next to a purple ocean isn't uniquely interesting.
Or another example would be the radiant AI from Skyrim that could automatically generate quests for the players.
I think that using an LLM to model NPCs runs into the same problem(s). In the end, there are two cases: either the behavior is constrained enough to keep the game on the rails, and thus the randomness in the dialogue only ads some flavor but there isn't enough freedom to generate new quests and directions for the story. In that case, the added space to explore really doesn't change the nature of the game or add much.
In the second case, you let the model go off the rails and have a harness around it that generates a world matching the hallucinated responses, which would allow an LLM to dynamically generate quests and such, but then the design of your game is subject to being compromised by the randomness of an LLM. E.g. it's not just Red Dead Redemption 3.0 with some funny characters, sometimes it's a historical game and other times aliens show up.
Maybe that's compelling to some people but I've done acid before and really don't need all my media to recreate that sensation of reality drifting.
Honesty, in any game quests feel artificial, whether they've been generated by humans or AI.
Try Rain World:
https://rainworldgame.com/
Not an RPG and no quests but it has some of the most well-done game AI in any game I have ever played. Basically every mob in the game has its own goals and goes about fulfilling them regardless of what the player is doing, and where the player is etc. It's more like a simulated environment and the game drops you in the middle of it and you have to learn how the world works, navigate and survive it.
It's not even any kind of advanced technology, just how fun game AI can be if a developer gives it sufficient attention, instead of basically bodging together some behaviour trees and calling it a day while spending all the compute and development budget for graphics. That's one reason why game AI sucks so much most of the time.
Another reason is that players themselves don't want convincing enemies in a convincing environment. I always recall a review of Rain World where the game critic threw his controller because he thought the enemies respawn randomly when the player dies, thus depriving the game critic from the opportunity to memorise their "patterns". In truth, the enemies don't "respawn" they just go about their business while the player is regenerating and so they're not always where you last left them when you return at the spot you died. The world keeps turning when you're gone. So you do have to learn their "patterns" except those patterns are not trivial patrol cycles but actual, you know, behaviours.
But, no. That won't do. Give us a game where we can memorise every telegraphed attack so we know when to press which button with millisecond precision like a mindless automaton.
Sorry, little rant there. I'm saying that many gamers and many devs don't actually want decent, convincing game AI.
Someone's already built #1. I've seen the demo, and it had a wow factor, but ultimately I don't think it'll revolutionize games.
Would Skyrim be better if you could talk to ever guard about what they had for breakfast? Would you ever be able to shake the knowledge that it's just an LLM pretending?
I'm not sure how best to put this. I think for me at least, I get the most enjoyment out of discovering my way through a story that somebody else wrote. This is maybe why I don't like multiplayer games as much a single player experiences. I want another human to tell me a story, and I love the feeling of uncovering the little pieces of story and wondering if I've got it all and how much more there is. If an LLM is just randomly making it up as it goes, I'm not discovering anything. I'm not hearing a story. Instead, I'm just having a transient conversation with myself.
I guess it's equivalent to the difference between visiting an art gallery, versus watching a computer generate fake paintings. One has human intent behind it and that makes it compelling, the other is soulless and empty
> I love the feeling of uncovering the little pieces of story and wondering if I've got it all and how much more there is
To me, this is great too, and it's why I still enjoy reading fiction.
However, having conversational NPCs which can behave in complex ways isn't just about a story any longer, it's about a more accurate simulation. I want to see where the non-deterministic machine(s) can take me. I want to poke and prod and discover new mechanisms. I want to make my own story.
Once I was quite religious. I was discouraged from reading fiction by elders because they said it is a "waste of time" and "basically lies". In this way, whether it was written by a human or a machine is irrelevant if I can't tell the difference. I would have a qualitatively different response if I knew whether a human or bot wrote it, but in the long run I'm not sure it matters in cases where I don't know.
>I want to make my own story
Then open up a word document and write one! Don't outsource that desire to a ball of linear algebra.
1. Is not a gamer's dream. It's terrible and you'll find out quite fast you're not interested in everyone's background and scream to most NPC's to shut the fuck up and get to the point.
It's just as terrible as injecting 'realism' in games for the sake of 'realism'.
Agree, I'm not at all looking for #1, at all. Good dialogue is an art form.
I agree and I think games are ruined by dialog and quests. I like procedurally generated worlds, not stories, but I want the worldgen algorithm to be written by a human, crudely and with idiosyncrasies. I do not want to wander around a world of blandly plausible filler material.
2 replies →
Presumably the art in a game like that would consist in setting up the world and prompts to make the AI NPCs interesting.
> It's terrible and you'll find out quite fast you're not interested in everyone's background and scream to most NPC's to shut the fuck up and get to the point.
Many of the interactions in RDR2 are quite mundane, and despite thousands of hours of (high quality) voice acting, it can become quite repetitive.
I could very much see those micro-interactions being LLM generated, but the TTS would need to be a step above where even the best models are now to come close to RDR2s production quality.
The repetitiveness in video game dialogues is a feature, not a bug. Amongst other things, it allows you to re-retrieve information and hone in faster on what’s story and what’s relevant progression. Having each character invent their own inconsistent sloppy backstory whenever you talk to them is not a positive, it’s not good immersion when every character is a chatbot that can inadvertently give you story beats you shouldn’t be aware of yet or you missed some crucial bit of information but no one talks about it anymore (or worse, never did). In that world, those games would be made popular by people breaking the LLMs in funny ways, not the gameplay itself.
9 replies →
#2 has been fiction for all but 0.1% or less of authors for many years.
As of a few years ago - before AI writing was an issue - the average full time author in the UK would have earned more flipping burgers (but their household incomes are above average - it's a middle class hobby for most).
And only a miniscule proportion of authors are full time.
#1 is a marketer at AAA studio's dream, not a gamer's dream. People consuming works of art appreciate quality of storytelling and immersiveness.
#1 is rather what unexperienced game developers think what is a gamer's dream. It isn't---in fact, unlimited freedom is a recipe for confusion.
"unlimited freedom is a recipe for confusion" Minecraft, no mans sky, dwarf fortress etc etc all disagree.
When I know that #1 has been generated by an AI tool, I really lose interest in whatever backstory the character is supposed to have.
Writing in video games is often pretty bad, but at least we used to know / sometimes can know it was done by a guy/gal in an office somewhere, trying to create something interesting. Now it’s just an algorithm.
I’m a gamer, #1 is not my dream. Games, as with any other work of art, are also an exercise in curation on the part of the developers. Without that filter, and that common experience with other players, I might as well scroll reels and get an equivalent experience.
I wrote a text-based adventure game engine that implemented quite a detailed world model. The results were very engaging indeed, but after a time I realised that while its game world was hugely detailed to the point of individual characters having their own simulated thoughts, feelings and world view, the game was quite shallow in terms of conceptual depth - it very successfully rehashed genre cliches, but nothing about it felt new or fresh.
Did you publish this game somewhere?
No, it's still sitting on my machine. It was just a quick-and-dirty shim on top of an LLM service, and not worth maintaining. And it ate tokens at an amazing rate updating every character's point of view.
> each NPC is voiced by AI and can respond to you in a convincingly human fashion
This is no longer fiction - see the latest AI update of PUBG.
#1 is Dungeons and Dragons, except for the word 'video' game.
#1 is as much a gamer's dream as Youtube shorts AI slop is a cinephile's dream.
I don’t get your ambivalence, when you seem to understand that the negatives of #2 far outweigh the positives[1] of #1. That’s something that has always been really weird in these LLM discussions, it’s like that Tom Toro cartoon:
https://www.newyorker.com/cartoon/a16995
[1]: And even those are subjective. I wouldn’t want that, and the other replies so far agree that would be bad.