Comment by panki27

Have you seen Mantella yet? https://art-from-the-machine.github.io/Mantella/

tl;dr: let's you talk to all NPCs via LLM. Last time I tried it, latency was too much for it to be enjoyable, your approach seems to be much faster.