← Back to context

Comment by d4rkp4ttern

1 month ago

This is not strictly speech-to-speech, but I quite like it when working with Claude Code or other CLI Agents:

STT: Handy [1] (open-source), with Parakeet V3 - stunningly fast, near-instant transcription. The slight accuracy drop relative to bigger models is immaterial when you're talking to an AI. I always ask it to restate back to me what it understood, and it gives back a nicely structured version -- this helps confirm understanding as well as likely helps the CLI agent stay on track.

TTS: Pocket-TTS [2], just 100M params, and amazing speech quality (English only). I made a voice plugin [3] based on this, for Claude Code so it can speak out short updates whenever CC stops. It uses a non-blocking stop hook that calls a headless agent to create the 1/2-sentence summary. Turns out to be surprisingly useful. It's also fun as you can customize the speaking style and mirror your vibe etc.

The voice plugin gives commands to control it:

    /voice:speak stop
    /voice:speak azelma (change the voice)
    /voice:speak <your arbitrary prompt to control the style or other aspects>

[1] Handy https://github.com/cjpais/Handy

[2] Pocket-TTS https://github.com/kyutai-labs/pocket-tts

[3] Voice plugin for Claude Code: https://github.com/pchalasani/claude-code-tools?tab=readme-o...

15 comments

d4rkp4ttern

Reply
raajg  1 month ago

I've been dabbling with STT quite a bit and built my own tool using Deepgram. But just tried Handy and it's SO FREAKING FAST! Love it.

  • d4rkp4ttern  12 days ago

    Hex is my new favorite STT on MacOS. Also uses Parakeet V3. I didn't think it could possibly be faster than Handy, but it is much faster - even long ramblings transcribed within a second. It's MacOS only, leverages the CoreML / Apple Neural Engine.

    https://github.com/kitlangton/Hex

    Also the transcriptions with hex don't seem to suffer from some of the issues with Handy, such as stutter.

  • srinifromsalem  12 days ago

    For local speech-to-text, Whisper remains the gold standard - you can run it locally with good accuracy across languages. For speech-to-speech, you'd typically chain Whisper with a local TTS model like Coqui TTS or use something like Tortoise TTS for higher quality but slower processing. The key is balancing accuracy, speed, and resource usage based on your specific use case. If you're doing content creation workflows, consider what post-processing you might need - sometimes the raw transcription needs structure and enhancement beyond just accurate words.

    • mumbleflow  11 days ago

      +1 on the post-processing point. Raw Whisper output is ~90% there but punctuation, grammar, and formatting are the missing piece.

      I built MumbleFlow to address exactly this — whisper.cpp for STT plus llama.cpp for smart text cleanup, all running on-device. Metal/CUDA accelerated, sub-second latency on Apple Silicon. Global hotkey works in any app.

      $5 one-time, no cloud, no subscription. https://mumble.helix-co.com

  • d4rkp4ttern  1 month ago

    Yes especially with Parakeet V3. It’s also nicely hackable, I Clauded a couple PRs to improve the experience, like removing stutters and filler words.

freakynit  1 month ago

A 25MB TTS model: https://github.com/kittenml/kittentts

  • d4rkp4ttern  1 month ago

    Nice, I’ll have to try it out. They should really make a uv-installable CLI tool like pocket-TTS did. People underestimate just how much more immediately usable something becomes when you can simply get something by doing “uv tool install …”

    • freakynit  1 month ago

      True that. People, especially developers, underestimate the importance of packaging. Or, in general, making it easier for others to use your product.

    • d4rkp4ttern  1 month ago

      So I benchmarked it and there’s really no advantage over pocket TTS. There are some tradeoffs like Kitten doesn’t have streaming audio.

indigodaddy  1 month ago

Hi, so I'm looking for an stt that can happen on a server/cron, that will use a small local model (I have 4 vCPU threadripper CPU only and 20G ram on the server) and be able to transcribe from remote audio URLs (preferably, but I know that local models probably don't have this feature so will have to do something like curl the audio down to memory or /tmp and then transcribe and then remove the file etc).

Have any thoughts?

3dsnano  1 month ago

posts like this are why i visit HN daily!!!

thanks for sharing your knowledge; can’t wait to try out your voice plugin