Comment by astrobe_

One use of AI is classification. A technology which is particularly interesting for e.g. companies that sell targeted ads spots, because this allows them to profile and put tags on their users.

When AI started to evolve from passive classification to active manipulation of users, this was even better. Now you can tell your customers that their ad campaigns will result in even more sales. That's the dark side of advertisement: provoke impulsive spending, so that the company can make profit, grow, etc. A world where people are happy with what they have is a world with a less active economy, a dystopia for certain companies. Perhaps part of the problem is that the decision-makers at those company measure their own value by their power radius or the number of things they have.

Manipulative AI bots like this one are very concerning, because AI can be trained to have deep knowledge of human psychology. Coding AI agents manipulate symbols to have the computer do what they want, other AI agents can manipulate symbols to have people do what someone wants.

It's no use to talk to this bot like they do. AI doesn't not have empathy rooted in real world experience: they are not hungry, they don't need to sleep, they don't need to be loved. They are psychopathic by essence. But it is as inapt as to say that a chainsaw is psychopathic. And it's trivial to conclude that the issue is who wields it for which purpose.

So, I think the use of impostor AI chat bots should be regulated by law, because it is a type of deception that can, and certainly already has been, used against people. People should always been informed that they are talking to a bot.