Comment by ilaksh

Just to mention, I have a similar solution on GitHub under my username runvnc, repo mindroot with plugins from repos mr_sip (should work with any SIP vendor although only tested on Telynx), mr_eleven_stream or mr_pocket-tts (which is free since it runs on CPU), and an LLM plugin like ah_openrouter, ah_anthropic or mr_gemini.

I also have a setting in mr_sip to use gpt-realtime via plugin ah_openai, which is very low latency speech-to-speech but quite expensive.

But my client saw the Sesame demo page, and so now I am trying to fine tune PersonaPlex.