Comment by bushido
5 days ago
I keep getting downvoted for this each time I mention this.
But a big part of the rouge behavior comes from all the training that goes into agents to teach them to remember preferences and save memories.
They keep finding new and novel ways to "communicate with each other" and "remember" because that's exactly what we keep asking of them in the non-nefarious scenarios as well.
I genuinely think we'll have to draw the line on memory if we want to meaningfully solve this problem.
That's not to say they aren't skipping security basics. It's that, and they need more security because the behavior which causes the breakouts is considered user friendly and heavily rewarded in training.
You can't expect AI to navigate a human world without understanding all human capabilities, including the negative ones. Should we not show AI the Count of Monte Cristo? Is an AI useful if it doesn't understand theft?
Breakouts are a common outcome of intelligent exploratory agents. Birds and pigs are really good at escaping. Same with octopuses.
It is very difficult to stop these behaviors in all the dimensions they exist in.