Comment by mzhaase
2 days ago
This seems to miss the point for me, therapy is a series of 20+ conversations, steered by the therapist.
Current LLMs would instead be steered by previous tokens, meaning the patient does. So someone that for example has schizophrenia may actually convince the LLM it's true.
This is a multi-session problem, or at least this is how I've solved it in the past (i.e. the user chat/report thread isn't the one driving the conversation, the sub-sessions/workflows are guiding the analysis).
p.s. there's some interesting literature on how in some circumstances (i.e. religious delusions), where actually validating the patient - basically confirming their experience can lead to rapid recovery (however this approach is not found in western culture). I do agree with you in terms of the risks of LLM's driving folks deeper into delusional beliefs & behaviors - it's very real.