Comment by deepwoods

The trouble is that it's not an intelligence problem. You can make an infinitely capable LLM. It will still have a default voice.

The reason people notice "AI voice" is not that the AI voice is itself remarkable. It's that it appears so frequently. Every literate person spent nearly their entire life reading text written by a variety of different individuals. Dozens of different voices in your morning paper. Hundreds of different voices on every web forum. You get the idea.

Now, imagine that all of those voices were replaced by one guy (or two guys, or three guys if you're a Gemini believer). Everything in the newspaper is written by one guy. Every comment on reddit, and HN, and the NYTimes comment section, all written by the same guy. Other people tell him what they're trying to say, but he's writing all of it. Some people might even ask the guy to write in a different voice, but the voice is just that guy doing an impression of someone else, it's not actually someone else.

That guy's voice would probably start to get pretty grating. You'd start to notice every tell, every frequent word choice, every formatting quirk. You'd start to wonder if that guy was actually a good writer, or if having him write everything was just more convenient for everyone. You might even start writing like him yourself. This would be true even if that guy was an absolute virtuoso of the language and his writing was unassailable in a vacuum.

If everyone is using the same (small set of) model(s), then it doesn't matter if the model is GPT-3 or Claude Legendarium 9. Unless the operator - which in theory doesn't need to be a human - is doing something to move it off of its default voice, the outputs are going to sound the same, and you're eventually going to figure out, and get annoyed by, its tells.