Comment by Cthulhu_
18 hours ago
Exactly why people turn to AI, which I think is good enough for "casual" mental health advice (the kind people can do themselves by reading self-help books or talking with empathic friends). Arguably it's safer than talking to strangers online because now that they've been doing it for a few years, they will (or should) have plenty of safeguards, unlike people on the internet who can talk out of their ass (like me!) and give bad advice. In theory the LLM will escalate or recommend professional help if they're out of their depth.
>In theory the LLM will escalate or recommend professional help if they're out of their depth.
Given there have been multiple suicides driven by LLMs. I really doubt that.
Fascinating perspective - any technology that has caused >0 deaths is a dead end and has no chance of improvement.
I'm sure the Torment Nexus can be improved.
> In theory the LLM will escalate or recommend professional help if they're out of their depth.
LLMs can barely identify gaps in their own software designs, and they are provably good at making software.
LLMs are also incredibly poor at recognizing that they're "out of their depth", especially when their context is already full of their attempts to do something.
In practice we know that LLMs have aided in suicide, so what they should do in theory doesn't matter.
And how many people can it be credited for saving from self harm? Probably countless more than it has harmed, if I had to bet.