Comment by zaptrem
1 day ago
> Additionally, rare hallucinations in Voice Mode persist with this update, resulting in unintended sounds resembling ads, gibberish, or background music. We are actively investigating these issues and working toward a solution.
Would be cool to hear some samples of this. I remember there was some hallucinated background music during the meditation demo in the original reveal livestream but haven't seen much beyond that. Artifact of training on podcasts to get natural intonation.
I use advanced voice a lot and have come across many weird bugs.
1) Every response would be normal except end with a “whoosh” like one of those sound effects some mail clients use when an message is sent, and the model itself either couldn’t or wouldn’t acknowledge it.
2) The same except with someone knocking on a door. Like someone would play on a soundboard.
3) The entire history in the conversation disappearing after several minutes of back and forth, leading to the model having no idea what I’m talking about and acting as if it’s a fresh conversation.
4) Advanced voice mode stuttering because it hears its own voice and thinks it’s me interrupting (on a brand new iPhone 16 Pro, medium-low built in speaker volume and built-in mic).
5) Really weird changes in pronunciation or randomly saying certain words high-pitched, or suddenly using a weird accent.
And all of this was prior to these most recent changes.
It also stutters and repeats sometimes and says poor connection even though I know the connection is near-ideal.
I may know why that first one happens! They’re not correctly padding the latent in their decoder (by default torch pads with zeros, they should pad with whatever their latent’s representation of silence is). You can hear the same effect in songs generated with our music model: https://sonauto.ai/
Yeah we’re too lazy to fix it too
I’m super curious now, how does padding lead to repeatedly ending tts replies with what seem to be an actual non-speech sound effect?
3 replies →
> 4) Advanced voice mode stuttering because it hears its own voice and thinks it’s me interrupting
I experience the same issue on an iPhone 15 Pro Max and have to mute the mic whenever I'm listening to a response. I wish they added an option to disable voice interruptions so that it could be interrupted only by touch.
Oh man, I thought the “whoosh” was an intentional indicator that it was done speaking.
If anyone's wondering, here's a short sample. It quietly updated last night, and I ended up chatting for like an hour. It sounds as smart as before, but like 10x more emotionally intelligent. Laughter is the biggest giveaway, but the serious/empathetic tones for more therapy-like conversations are noticeable, too. https://drive.google.com/file/d/16kiJ2hQW3KF4IfwYaPHdNXC-rsU...
Did it really say partwheel or is it garbled?
It sounded like partwheel to me too. Now I wonder if more people will hear it as partwheel due to the McGurk effect [0].
[0] https://en.wikipedia.org/wiki/McGurk_effect
Holy moley. Thanks for sharing. I had to work with the API version a lot the last week and it was frustrating how "old" it felt intelligence-wise. This is in another league, I hope it's 4.1 x audio training, I'd love to talk to this. Current one is passable for hands-free RAG, that's it for me.
they still need to post-train out the emissions of all the trapped souls