Comment by antics
3 days ago
Why always nuclear weapons in these conversations, lol. Is there something in this community that makes this example particularly interesting?
Go read the transcripts where the AI is “unaligned” and has some very bad impact, like encouraging kids to kill themselves. There is not some obvious piece of knowledge to remove to make it safe. You’d have to basically lobotomize the model to make those conversations impossible. People won’t use a model that’s lobotomized.
I disagree: You could lobotomize all the social-interaction parts of a model and it would still be very useful for practical purposes. Personally, I think it would be wonderful if we had models that knew how to debug javascript or look up a missing shipment at the warehouse, but didn't know how to pretend to be someone's friend.
If, as you suggest, fewer humans find it compelling to spend hours talking to a machine, so much the better.
There are maybe examples where your point stands but that’s probably the worst one. The point of that knowledge is actually to answer questions. That is where social skills in the model come from: training it to answer questions.
I don't think a model needs to be a parasocial companion in order to answer practical questions effectively. It doesn't need knowledge that allows it to mimic empathy, or simulate emotional engagement to answer "how can I repair this lawnmower?" or "why is this SQL query slow?"