Comment by MyOutfitIsVague
2 days ago
Psychosis is not a binary, and there are strong genetic components, but it's not the entire story. People develop psychoses where there weren't any before all the time, often but not necessarily in response to trauma. Some people are more predisposed to it than others.
Even if you hold that you need that genetic predisposition, and that a perfectly sane person can never become "insane" as a result of an experience (not a position I agree with), there is still the very real possibility that many people with the predisposition would never have had this "already insane" condition ever triggered or exposed. Think about the suicide cults that have formed, notably Jonestown. It's easy to consider the mass suicide (and murder of the members' own children) as "insane" behavior. What are the odds that every single person was genetically insane? Comparatively, what are the odds that an extremely persuasive and charismatic personality can rewire a person's reasoning to believe something "insane"?
If you think of ChatGPT as capable of being an automated Jim Jones, I think the danger becomes more clear. These tools are not incapable of being dangerous.
> If you think of ChatGPT as capable of being an automated Jim Jones, I think the danger becomes more clear. These tools are not incapable of being dangerous.
Thinking back on the documentaries about NXIVM, personas like Keith Raniere also seem eerily similar to LLMs in the sense that you can give them any prompt on the spot, and they will start generating long speeches which are essentially bullshit but on the surface seem coherent or even deep.
Another danger, that I guess we just have to live with now, is not just that ChatGPT could Jim Jones people, but that someone who didn't have the natural charisma of a cult leader could use ChatGPT on purpose to Jim Jones people. It may not work so well in person, but online ... yikes.
Libraries also contain dangerous information.
I didn't say anything about "dangerous information". You may have misinterpreted the point I was making. It's not about dangerous information, it's about a personality that is very persuasive, can be wildly charismatic, and can convince people to do dangerous things they wouldn't otherwise do.
People don't join cults because of "dangerous information".