Comment by bflesch
4 hours ago
A lot of effort is spent to make the "conversation" feel just like a human-to-human interaction. This is not a naturally occurring phenomenon due to the technology, but rather a feature carefully engineered by those companies in order to get people hooked. Then they have all these tiny nudges like the typing animations or the "thinking..." popups before the next chat message appears.
At some point you might have also noticed the over-use of emojis, the bolted-on jokes, and the tendency to always approve what the user says (even though they have toned that down after backslash). At some point too many people thought they were in a relationship with the chatbot, because it always encouraged and approved them, so they had to hotfix it.
It's a bunch of really dark psychological patterns that are carefully combined by very clever people in order to create the false illusion that the user is experiencing something deeper than an engineered simulation of human interaction.
I think the technology is really useful, but they are obviously not happy with simply replacing a google-like query interface, they want users to fall in love with the product and mentally treat it like a fellow human being - and that's what I think is insincere.
To get more concrete are you using coding agents like Claude Code/ Codex/ opencode etc? What kind of work are you doing specifically?
If you are doing the kind of median enterprise tech work these tools are just good enough to do it at a relatively high level or atleast heavily augment people doing it.
Examples would be like adding routine CRUD features to APIs/ improving observability/ adding tests or accessibility features to codebases etc.
POV OpenAI, early 2021. You have a pretty good next-token predictor called GPT-3. You noticed sometimes it can do useful things if you write out the start of a task and let it predict the next step. However, sometimes it's very difficult to frame a task that way. So instead you train it to predict the answer based on a question or instructions. Oh wait, it didn't get it right the first time... better let users iterate too. Now you have a conversation.
Things like loading indicators are basic good UI dating back to the 90s.
A/B testing and generally following user preferences might still push towards the dynamic you're describing, as it did with gpt-4o. xAI and a few other companies like Replika also intentionally created "companion"/porn AIs. But in general, natural language was previously exclusive to humans. It's completely natural that the first technology capable of it would therefore be perceived as more human. It's worth trying to resist this tendency, but it doesn't require evil intent on the part of the creators.
>A lot of effort is spent to make the "conversation" feel just like a human-to-human interaction.
We'll in humans we call this an education and it takes quite a long time to get one.