Comment by alabastervlog

My wife's job is doing something similar, but without the API (not exactly a game, but game-adjacent)

I'm fairly sure their approach is going to collapse under its own weight, because LLM-only is a testing nightmare, and individual people writing these things have different knacks and styles that affect the entire interaction, so getting someone to come in and fix one that someone wrote a year ago but now they're not with the company is often going to approach the cost of re-doing it from scratch. Like, the next person might just not be able to get the right kind of behavior out of a session that's in a certain state, because it's not how they'd have written it into that state in the first place so they have trouble working with it, or the base prompt for it is not an approach they're used to (but if they touch it, everything breaks) and they'll burn just so very much time on it. Or they fix that one part that broke, but in a way that messes up subsequent interactions. Used this way, these things are fragile.

Using it to translate text into API calls and back is so much more sane.