Comment by thelisu

3 days ago

The first goal is to align the model to the least common denominator of ethics. Even if there is diversity in values, everyone agrees that murder is bad, or that you shouldn't torture your neighbor for fun.

The whole point of the thought experiment is that not everyone agrees that pulling the lever to kill the one man is murder and thus immoral.