Comment by quitit

> But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want.

This seems tautological to the point where it's meaningless. It's like saying that if you try to hire an employee that's going to challenge you, they're going to always be a sycophant by definition. Either they won't challenge you (explicit sycophancy), or they will challenge you, but that's what you wanted them to do so it's just another form of sycophancy.

To state things in a different way - it's possible to prompt an LLM in a way that it will at times strongly and fiercely argue against what you're saying. Even in an emergent manner, where such a disagreement will surprise the user. I don't think "sycophancy" is an accurate description of this, but even if you do, it's clearly different from the behavior that the previous poster was talking about (the overly deferential default responses).

Hmm. I think you may be confusing sycophancy with simply following directions.

Sycophancy is a behavior. Your complaint seems more about social dynamics and whether LLMs have some kind of internal world.

>> That's the default chatbot behavior. Many of these people appear to be creating their own personalities for the chatbots, and it's not too difficult to make an opinionated and challenging chatbot, or one that mimics someone who has their own experiences. Though designing one's ideal partner certainly raises some questions, and I wouldn't be surprised if many are picking sycophantic over challenging.

> You can make an LLM play pretend at being opinionated and challenging. But it's still an LLM. It's still being sycophantic: it's only "challenging" because that's what you want.

Also: if someone makes it "challenging" it's only going to be "challenging" with the scare quotes, it's not actually going to be challenging. Would anyone deliberately, consciously program in a real challenge and put up with all the negative feelings a real challenge would cause and invest that kind of mental energy for a chatbot?

It's like stepping on a thorn. Sometimes you step on one and you've got to deal with the pain, but no sane person is going to go out stepping on thorns deliberately because of that.

I lean toward the opinion there are certain things people (especially young people) should be steered away from because they tend to snowball in ways people may not anticipate, like drug abuse and suicide; situations where they wind up much more miserable than they realize, not understanding the various crutches they've adopted to hide from pain/anxiety have kept them from happiness (this is simplistic, though; many introverts are happy and fine).

I don't think I have a clear-enough vision on how AI will evolve to say we should do something about it, though, and few jurisdictions do anything about minors on social media, which we do have a big pile of data on, so I'm not sure it's worth thinking/talking about AI too much yet, at least as it relates to regulating for minors. Unlike social media, too, the general trajectory for AI is hazy. In the meantime, I won't be swayed much by anecdotes in the news.

Regardless, if I were hosting an LLM, I would certainly be cutting off service to any edgy/sexy/philosophy/religious services to minimize risk and culpability. I was reading a few weeks ago on Axios of actual churches offering chatbots. Some were actually neat; I hit up an Episcopalian one to figure out what their deal was and now know just enough to think of them as different-Lutherans. Then there are some where the chatbot is prompted to be Jesus or even Satan. Which, again, could actually be fine and healthy, but if I'm OpenAI or whoever, you could not pay me enough.

Excellent point. It’s bad for humans when humans do it! Imagine the perfect sycophant, never tires or dies, never slips, never pulls a bad facial expression, can immediately swerve their thoughts to match yours with no hiccups.

It was a danger for tyrants and it’s now a danger for the lonely.

I wonder if in the future that'll ever be a formal medical condition: Sycophancy poisoning, with chronic exposure leading to a syndrome of some sort...

That explains why Elon Musk is such an AI booster. The experience of using an LLM is not so different from his normal life.

Hey hey safe injecting rooms have real harm minimisation impacts. Not convinced you can say the same for chatbot boyfriends.

That's exactly right, and that's fine. Our society is unwilling to take the steps necessary to end the root cause of drug abuse epidemics (privatization of healthcare industry, lack of social safety net, war on drugs), so localities have to do harm reduction in immediately actionable ways.

So too is our society unable to do what's necessary to reduce the startling alienation happening (halt suburban hyperspread, reduce working hours to give more leisure time, give workers ownership of the means of production so as to eliminate alienation from labor), so, ai girlfriends and boyfriends for the lonely NEETs. Bonus, maybe it'll reduce school shootings.

Given that those tend to have positive effects for the societies that practice this is that what you wanted to say?

> Even "the dating scene is terrible" is human interaction.

For some subset of people, this isn't true. Some people don't end up going on a single date or get a single match. And even for those who get a non-zero number there, that number might still be hovering around 1-2 matches a year and no actual dates.

Swiping on thousands of people without getting a single date is not human interaction and that's the reality for some people.

I still don't think an AI partner is a good solution, but you are seriously underestimating how bad the status quo is.

We do see - from 'crazy cat lady' to 'incel', from 'where have all the good men gone' to the rapid decline of the numbers of 25-year-olds who have had sexual experiences, not to mention from the 'loneliness epidemic' that has several governments, especially in Europe, alarmed enough to make it an agenda pointt: No, they would not. Not all of them. Not even a majority.

AI in these cases is just a better 'litter of 50 cats', a better, less-destructive, less-suffering-creating fantasy.

Not all human interaction is a net positive in the end.

In this framing “any” human interaction is good interaction.

This is true if the alternative to “any interaction” is “no interaction”. Bots alter this, and provide “good interaction”.

In this light, the case for relationship bots is quite strong.

The point is you wouldn't deal with people. Every interaction becomes a transaction mediated by an AI that's designed to make you happy. You would never genuinely come in contact with other perspectives; everything would be filtered and altered to fit your preconceptions.

It's like all those dystopias where you live in a simulation but your real body is wasting away in a vat or pod or cryochamber.

don't worry, "how is babby formed" is surely in every llm training set

It could be the case that society is responding to overpopulation in many strange ways that serve to reduce/reverse the growth of a stressed population.

Perhaps not making as many babies is the longterm solution.

Wait, how did this work in The Matrix exactly?

Decanting jars, a la Brave New World!

[dead]

It seems we're not quite there, yes. But you should have seen the despair when GPT 5 was rolled out to replace GPT 4.

These people were miserable. Complaining about a complete personality change of their "partner", the desperation in their words seemed genuine.

One difference between "AI chatbots" in this context and common flight simulator games is that someone else is listening in and has the actual control over the simulation. You're not alone in the same way that you are when pining over a character in a television series or books, or crashing a virtual jumbo jet into a skyscraper in MICROS~1 Flight Simulator.

This is the exact kind of thinking that leads to this in the first place. The idea that a human relationship is, in the end, just about what YOU can get from it. That it's just simply a black box with an input and output, and if it can provide the right outputs for your needs, then it's sufficient. This materialistic thinking of other people is a fundamentally catastrophic worldview.

A meaningful relationship necessarily requires some element of giving, not just getting. The meaning comes from the exchange between two people, the feedback loop of give and take that leads to trust.

Not everyone needs a romantic relationship, but to think a chatbot could ever fulfill even 1% of the very fundamental human need of close relationships is dangerous thinking. At best, a chatbot can be a therapist or a sex toy. A one-way provider of some service, but never a relationship. If that's what is needed, then fine, but anything else is a slippery slope to self destruction.

Yes, great comment.

What do you think of the idea that people generally don't really like other people - that they do generally disappoint and cause suffering. (We are all imperfect, imperfectly getting along together, daily initiating and supporting acts of aggression against others.) And that, if the FakePeople™ experience were good enough, probably most people would opt out of engaging with others, similar to how most pilot experiences are on simulators?

Disturbing and sad.

> Maybe we should not want to get prepared for RealPeople™ if all they can do is break us and disappoint us.

Good thing that "if" is clearly untrue.

> AI chatbots as relationship replacements are, in many ways, flight simulators:

If only! It's probably closer to playing star fox than a flight sim.

[flagged]

Nah, most circles of neurodivergent people I've been around have humility and are aware of their own fallibility.

The comment seems less clearly-written (e.g., "It can happen for many multiple ways") than how a chatbot would phrase it.

We’re all just in a big LLM-generated self-licking-lollipop content farm. There aren’t any actual humans left here at all. For all you know, I’m not even human. Maybe you’re not either.