Show HN: Pinch – macOS voice translation for real-time conversations

3 days ago (startpinch.com)

Hey HN! I’m Christian, daily lurker and some might remember our original launch post (https://news.ycombinator.com/item?id=42935355). Today we're launching Pinch for Mac, which we believe is a step-change improvement in real-time AI translation. Our vision is to make cross-lingual conversations feel as natural as regular conversations.

TL:DR During an online meeting, the app instantly transcribes and translates all audio you hear, and allows you to decide when you translate your voice and when you don't. It's invisible to others (like Granola), and works everywhere without any meeting bots. Try it at startpinch.com

Here's a live demo we recorded this morning, without cuts: https://youtu.be/ltM2p-SosLc

When we first launched Pinch, we shipped a video conferencing solution with a human-like AI interpreter that was an active participant in your call. Our users hold the spacebar down while speaking to the translator, and when they release the spacebar the translator speaks out to the entire room.

That design was intentional - it puts the task of context selection on the user and prevents people from interrupting each other awkwardly (only one person can press spacebar at a time). It also comes with heavy tradeoffs, namely:

* Latency - Up to 2x longer meeting lengths due to everyone hearing your full sentence and then the translation of your full sentence

* Friction with first-time users - Customers using Pinch for external communication often meet with new people each time, and we've learned of several that send out an instruction doc pre-meeting on how to join and use translation in the Pinch call. Bad signal for our UX.

* Restricting our customers to those who are meeting creators

Benefits of the desktop app:

1. It creates a virtual microphone that you can use in any meeting app

2. Instant transcription+translation means you can understand what's going on in real-time and interrupt where necessary

3. Simultaneous translation - after you start speaking, the others will hear your translated audio as fast as we can generate it, without interrupting your flow.

Over the last months our focus has been on developing a model and UX to support high translation accuracy while automating context selection - knowing exactly when it has enough words to start the translated sentence. We’ve rolled this out to the desktop app first.

We're incredibly excited to go public beta today, you can give it a try at www.startpinch.com

Cheers, - Christian

It looks like whisper + google translate + google TTS. Typical "quality" for that stack, bad latency, no any privacy.

I'm developer of "Linguist", a browser extension for translation in browser, and I say you that nowadays it is possible to translate text locally in device. Linguist have embedded offline translator. The same with TTS and voice recognition.

All this features may run locally in-device, even in browser extension, but not in macOS application?

This product looks rather like a malware that will spy on users and then blackmail us or sell our conversations to email scammers for better targeting or anything.

Additionally it is interesting that Chinese and Korean languages is not supported. You just use cloud services, they are all support these languages, why you don't? Is it to fake something?

"12 hours translation per month" for $29. 12 hours it's about 6-12 meetings? Who is your audience then?

I encourage you to offer a “translation” option that involves simplification (e.g. Simple German) where the speech language X is translated in a simplified version of the language X. There’s value there.

  • Fun idea, are you thinking about a language learning use case?

    • That's also just a plain accesibility - levelling the situation where someone is less fluent in a given language. You could even translate from english to simple english!

      1 reply →

    • There are some complicated words in German. When simplified have a translation. But in German it’s just, German. Streichholzschächtelchen. When one could just easily say Streichholzer.

Just need to polish the solution... I experienced some crashes with zoom.

I am the IDEAL user, willing to pay a lot if this works. Count on me feedbacks.

So far:

good: - initial config as good, easy and very simple to feel secure

bad: - high cpu usage - zoom asked me to restart mic - could not make sure if the software works... Google meet has not voice testing loop... Zoom has it, but it did not work for me.

Does this work with less-than-perfect audio, for example, watching foreign TV with background music?

  • Yes, if you're watching TV on your computer the app should translate all the speech. We haven't done any source separation work (musical singing vs actual speech) so if there's music it may pick up some of the lyrics.

Hey, this is very cool! I just tried to make something alike for Windows yesterday for talking in French (Bonjour le Québec). After a few hours fidgeting with Whisper, I ended up finding https://github.com/SakiRinn/LiveCaptions-Translator , but it does not have the virtual microphone feature. Do you plan to support Windows any time soon?