Comment by ForHackernews
3 days ago
> And if that is true, then it does not seem hopeful that there would be a way to prevent the output they are worried about, other than just removing a bunch of the knowledge. So maybe there is no alignment solution at all.
...wait, what? Removing a bunch of knowledge seems like a fantastic and obvious solution to me. My AI coding bot doesn't need to have knowledge about making nuclear weapons, the customer service AI doesn't need to have knowledge of dangerous viral synthesis techniques, my email receptionist bot doesn't need to know about C4.
If we can easily ablate dangerous knowledge from these models, we're sorted.
Why always nuclear weapons in these conversations, lol. Is there something in this community that makes this example particularly interesting?
Go read the transcripts where the AI is “unaligned” and has some very bad impact, like encouraging kids to kill themselves. There is not some obvious piece of knowledge to remove to make it safe. You’d have to basically lobotomize the model to make those conversations impossible. People won’t use a model that’s lobotomized.
I disagree: You could lobotomize all the social-interaction parts of a model and it would still be very useful for practical purposes. Personally, I think it would be wonderful if we had models that knew how to debug javascript or look up a missing shipment at the warehouse, but didn't know how to pretend to be someone's friend.
If, as you suggest, fewer humans find it compelling to spend hours talking to a machine, so much the better.
There are maybe examples where your point stands but that’s probably the worst one. The point of that knowledge is actually to answer questions. That is where social skills in the model come from: training it to answer questions.
1 reply →