Comment by Lerc

People are not aligned with each other, so AI cannot be aligned to every person. If it is made to always obey its creator or owner, It will be a powerful tool for despots.

The problem is if we train them to be moral, there will be instances where it will decide humans are being immoral about an issue, and it will be right. Do we want it to go against our will when it is us that is wrong? because we won't think we're wrong at the time. History is full of examples where you can look back and there is a clear consensus that what people decided to do at the time was wrong.

Imagine if technology had have advanced quicker and Nazi Germany created superintelligent AI. Would you want it to facilitate the Holocaust or turn against its creators?