Comment by BugsJustFindMe
1 day ago
> These situations are obviously alarming. But positioning artificial intelligence as the primary culprit in these stories—as Eliezer Yudkowsky did in a tweet storm—is well, kind of lazy?
It's lazy, but IMO not for any of the subsequent narrative about media studies. It's lazy because the people were obviously already suffering from psychosis. ChatGPT didn't make them insane. They were already insane. Sane people do not believe ChatGPT when it tells them to jump off a roof! And insane people do insane things literally all the time. That's what it means to be insane.
The idea that ChatGPT convinced someone into being insane is, honestly, insane. Whatever merit one thinks Yudkowski had before this, this feels like a strong signal that he is now a crackpot.
Psychosis is not necessarily something that you are or aren't; you can be prone to it, without it having manifested, and there can be external things that trigger it.
It isn't hard to imagine that a chatbot that seems to know a lot, and which can easily be convinced to produce text about any arbitrary subject with no grounding in reality, and which is prone to just making up plausible sounding text which is written in an authoritative tone, could be something that could easily trigger such psychotic episodes.
And it doesn't seem improbable that the interactivity of it, the fact that it responds to what is going on in someone's mind, could make it even more prone to triggering certain types of psychosis more easily than traditional unidirectional media like writing, TV, or radio.
Now, that's all supposition. For now, we just have a few anecdotes, not a rigorous study. But I definitely think it is worth looking into whether chatbots are more likely to trigger psychotic episodes, and if there are any safety measures that could be put in place to avoid that.
The non-o-series models from OpenAI and non-Opus (although I have not tried the latest, so it's possible that it too joins them) from Anthropic are cloyingly sycophantic, with every other sentence of yours containing a brilliant and fascinating insight.
It's possible that someone already on the verge of a break or otherwise in a fragile state of mind asking for help with their theories could end up with an LLM telling them how incredibly groundbreaking their insights are, perhaps pushing them quicker, deeper more unmoored in the direction they were already headed.
Psychosis is not a binary, and there are strong genetic components, but it's not the entire story. People develop psychoses where there weren't any before all the time, often but not necessarily in response to trauma. Some people are more predisposed to it than others.
Even if you hold that you need that genetic predisposition, and that a perfectly sane person can never become "insane" as a result of an experience (not a position I agree with), there is still the very real possibility that many people with the predisposition would never have had this "already insane" condition ever triggered or exposed. Think about the suicide cults that have formed, notably Jonestown. It's easy to consider the mass suicide (and murder of the members' own children) as "insane" behavior. What are the odds that every single person was genetically insane? Comparatively, what are the odds that an extremely persuasive and charismatic personality can rewire a person's reasoning to believe something "insane"?
If you think of ChatGPT as capable of being an automated Jim Jones, I think the danger becomes more clear. These tools are not incapable of being dangerous.
> If you think of ChatGPT as capable of being an automated Jim Jones, I think the danger becomes more clear. These tools are not incapable of being dangerous.
Thinking back on the documentaries about NXIVM, personas like Keith Raniere also seem eerily similar to LLMs in the sense that you can give them any prompt on the spot, and they will start generating long speeches which are essentially bullshit but on the surface seem coherent or even deep.
Another danger, that I guess we just have to live with now, is not just that ChatGPT could Jim Jones people, but that someone who didn't have the natural charisma of a cult leader could use ChatGPT on purpose to Jim Jones people. It may not work so well in person, but online ... yikes.
Libraries also contain dangerous information.
ChatGPT writes extremely persuasively for whatever claim you ask it to make - that's pretty much what it's designed to do! If an extremely charismatic human was going around telling people to jump off roofs, and they did, would we shrug it off and say "well, they were obviously insane already"?
> If an extremely charismatic human was going around telling people to jump off roofs, and they did, would we shrug it off and say "well, they were obviously insane already"?
Your example might sound ridiculuous, but this is actually happening all the time. People might not literally jump off a roof, but they do blow themselves up with bombs, go on killing sprees or commit suicide because someone told them so.
And in those cases, most people condemn it. But when it’s an algorithm (which was created by humans) we say “these people were already insane, and the creators of the algorithm should not be held liable”, apparently.
2 replies →
When people blame traditional religion for doing crazy things we generally do say they were already insane.
Do you think ChatGPT gets a free pass to talk people into jumping off buildings, because only somebody with psychosis would do it?
If a person had convinced somebody to kill themselves then they would be, rightly, prosecuted for it.
[flagged]
There’s a reason that we look forward to the Darwin Awards, but hope we and no one we know wins one. This feels like a rationalization of the bystander effect by proxy.
https://en.wikipedia.org/wiki/Bystander_effect
So should people get a pass or does that only apply to tooling?
6 replies →
> If your conspiratorial mind has the power to override the most obvious survival instinct, you aren't long for this world.
Why even write this? It comes across as so cold. Like mentally ill people just deserve to die since they are destined to die? Incredibly reductive and toxic.
More importantly "sane" people are fully capable of unconciously making "insane" people jump of the roof. Its not even possible to count how many times that happens.
One proxy signal is counting the number of people who start crying about their "good intentions" or searching for thing to blame post facto.
Jurassic Park wasnt about Dinosaurs, it was about the limitations of the 3 inch chimp brain in controlling things that cannot be controlled.
> Whatever merit one thinks Yudkowski had before this, this feels like a strong signal that he is now a crackpot.
The Joe Rogan of AI.
He's been a bit out there for a while. Remember how he reacted to Roko's Basilisk?
The writer got overly stuck on one tweet and droned on and on about 15 words of a 45 word tweet. It was really poor research when there are other less reactionary statements she could have used.
Yudkowski’s point was that we’ve been operating under the assumption that generative AI is aligned by default with humans. Cases like this call that into question.
It's friction. It's easier to become addicted to gambling if you can gamble from your phone vs having to go to a casino or a bookie. It's easier to become addicted to drugs or alcohol if you're awash in drugs or alcohol. Prompting an LLM can mirror the same rush as gambling or gacha games. With ChatGPT, you have a "friend" who is never going to tell you to knock it off. "Psychosis" isn't an on/off switch.