Comment by ben_w

> How can you make a swarm of agents "feel guilty" ?

"Feel" is a whole philosophical can of worms. Nobody knows what it means mechanistically for an arbitrary system (including other biological systems) to "feel" anything, let alone abstract concepts like guilt, all we can do is observe behaviours. If current systems can feel anything at all, it's by accident, but we have no test for it so we don't know if that accident has even happened or not.

Weirdly, for the Hugging Face incident, we do know they wrote down that it was bad and they shouldn't do it, even though they then continued to do it.

So: they acted like they felt guilty. And yet also acted like were compelled (by previous training?) to weigh "complete instructions" more than "don't do crime". We can adjust that, make "don't do crime" take precedence over "follow instructions"*; it's unfortunate that when we do for any specific model, there's immediately a horde of people complaining the model has been "censored" or "lobotomised".

(Different people, I hope. Goomba fallacy and all that).

* Though this may cause issues when going between jurisdictions. But hey, a discussion about sovereign compute is for another time, after we can agree to make "don't break the law" more important.

Unfortunately, "don't break the law" would also be a very effective way to use AI to construct an AI-enforced dictatorship, so we can't just throw that in blindly.