Comment by germinalphrase
1 day ago
Can you recommend any good resources that discuss structure and performance improvement of these types of systems?
1 day ago
Can you recommend any good resources that discuss structure and performance improvement of these types of systems?
Unfortunately, I don’t know of any.
Using LLMs for voice assistants is relatively new at scale that’s the difference between Alexa and Alexa+ and Gemini powered Google Assistant and what Apple has been trying to do with Siri for two years.
It’s really just using LLMs for tool calling. It is just call centers were mostly built before the age of LLMs and companies are slow to update
Understood. This overlaps with a side project where I’m getting acceptable (but not polished) results, so trying to do some digging about optimizations. Thanks!
One of my niches is Amazon Connect - the AWS version of Amazon’s internal call center. It uses Amazon Lex for voice to text. Amazon Lex is still the same old intent based system I mentioned. If it doesn’t find an intent, it goes to the “FallbackIntent” and you can get the text transcription from there and feed it into a Lambda and from the Lambda call a Bedrock hosted LLM. I have found that Nova Lite is the fastest LLM. It’s much faster than Anthropic or any of the other hosted ones.