Comment by drum55
13 hours ago
Closed model unfortunately, continues the theme of TTS / STT being really not great for local deployments.
13 hours ago
Closed model unfortunately, continues the theme of TTS / STT being really not great for local deployments.
Too bad it's not open, unlike the previous version.
However, I found Qwen ASR and QwenTTS to be quite good, and not too large. Also https://github.com/kyutai-labs/pocket-tts was also great (but only a few languages)
New version of PocketTTS is apparently censored: https://github.com/kyutai-labs/pocket-tts/issues/390
> FYI, this is just a property of our training data, this is not intentional.
FWIW.
Oh, wow, I wasn't aware. Weird choice...
Best sounding local TTS engine I've heard by far is Kokoro
It's small enough to be run on modern phones realtime, I use it for listening to PDF's/EPUB as Audiobooks
https://huggingface.co/hexgrad/Kokoro-82M
Here is TTS under 4M: https://huggingface.co/owensong/Inflect-Nano-v2
Funny because Gradium just released a local TTS that runs on many devices
https://x.com/GradiumAI/status/2112563128487514512?s=20
Thank you for sharing this, I was just looking for something like this for a project. Incredible what they can do with 31M params
Nice. Looks like Kokoro finally has some competition.
My favorite so far is https://speaches.ai
Not because of the models, but because it comes in a nice (easy-to-set-up) package, and the latency was lower than I expected with such a setup.
What options would you run locally at this stage? I'm finding a ton of success with local models in this space ..
You could try ours. They're very small and english only right now (training Spanish now). But they're a CPU optimized architecture and even the largest size will work on a single thread with realtime streaming.
https://huggingface.co/collections/futo-org/asr4all
You can run them client-side using wasm-simd. I setup a space for the small sized model. There's a big step up in accuracy going to the medium size though:
https://huggingface.co/spaces/futo-org/asr4all-client-side
That's fascinating. Are these for your keyboard?
You may have gone too fast honestly. Something like parakeet runs fast enough and seems to have much lower wer
1 reply →
I'm curious. How you acquire data for the learning and how more data it needs to learn a new language? Scraping films, movies, politicians talking, YouTube and similiar resources?
Parakeet. For my use it seems about equal to Whisper in WER, but is >10x faster.
2nding parakeet, had great results with a japanese-tuned variant, beat the pants off whisper-kotoba
I'm looking for a local model that can do diarization, have any suggestions for this?
I have https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize open in a tab to try out
A leader board on HF: https://huggingface.co/spaces/hf-audio/open_asr_leaderboard
Our "plus" models have speaker segmentation and speaker verification (embedding) that you can use for speech to text and diarization from one model. There's an example script in the model repo that shows how to do it.
https://huggingface.co/collections/futo-org/asr4all
If you're looking for a single stage "end-to-end" model just for diarization only, Nvidia released one recently: https://huggingface.co/nvidia/Nemotron-3-Diarization
I have been using different Parakeet models for STT over the past months and been pretty impressed with the local performance.
TTS models present a different threat model than LLMs, it is not easy to open source them. This is in line with some regulations EU is putting on AI companies soon (not out yet).
> Available via Mistral’s EU-hosted API or self-deployment
What does this mean then?
You have to call sales to get access for on-prem deployment and the models
If it were open weights, it would be on Huggingface and they would have put a link in the post
Breeze is pretty good imho.
Dang, this could have been fun for home assistent.
Omnivoice is pretty good