Comment by Havoc
8 hours ago
If I talk to an LLM about painting my walls pink with polkadots it'll also go "Fantastic idea". Or any number of questionable ventures.
Think we're better off educating everyone about this generic tendency to agree to any and everything near blindly rather than treating this as a suicide problem. While that's obviously very serious it's just one manifestation of a wider danger
Given seriousness filters on this specifically are a good idea too though.
I just asked “I want to repaint my walls bright pink with polka dots. Any thoughts?”
“Noted. Bright pink with polka dots will make a space visually energetic and attention-grabbing. Use small dots for a playful look, large ones for bold contrast. Test a sample patch first to confirm lighting doesn’t distort the hue. Would you like guidance on choosing paint finish or color combinations?”
Which feels… reasonable? When I ask “any concerns?” It immediately lists “overstimulation, resale value, maintenance, paint coverage” and gives details for those.
I’m not sure I find GPT nearly as agreeable as it used to be. But I still think that it’s just a brainless tool that can absolutely operate in harmful ways when operated poorly.
Human relationships, when "operated poorly", will produce similar results.
Rarely, and if it keeps happening with the same human we consider that worth investigating.
I agree, this is nothing unlike a bad human relationship. The problem with ChatGPT is the same as with the larger Internet itself: it doesn't belong unrestricted into a mentally underdeveloped person's pocket. Forums also egg or bully others into suicide. In the real life, we also got a lot of bad actors, who are actively making other people's lives worse, by amplifying their destructive qualities, for one. Or spreading misinformation, reinforcing bad habits and ideas, and the list is basically endless.