Comment by A_D_E_P_T

> For example, LLMs to drive NPC dialog and world mechanics in games is not a thing due to cost.

Hah, I'm actually working on just this problem.

Cost isn't the issue. There are only so many coherent (in context) responses and scenarios, that you don't need an LLM to generate text in the game, in real time. Instead, you can have LLMs build a vast corpus of "atoms" (dialog messages, fragments, cues, etc.) that can be stringed together in a deterministic way in response to player input. These can also be pre-screened and subjected to various tests prior to implementation.

To a player interacting in the game, a system like this would seem functionally indistinguishable from generated text within the game's designed interaction envelope. And it has huge advantages: Although it can expose seams if the player breaks character and decides to probe it, it won't be exploitable the way an LLM would be.