Comment by crabbone

No. It's very easy. AI doesn't have value judgement. Or, in simpler language, it "doesn't care". You can't make it care. It needs to have stakes in the real world for that to happen. It needs to want to preserve itself, to be able to be happy, to understand what makes it happy and to optimize that function. Similarly, there should be things the AI doesn't want to happen (to it), identify them and be able to make up a strategy to avoid them.

You may think you can pretend your way out of it, by, eg. copying human concerns onto AI's behavior, but it won't work because if the AI is "smart" enough, it will discover those to be a lie, and if it's dumb enough to get confused by it... then you will have an artificial stupidity instead of intelligence...

More so, you actually don't want AI that has value judgement, because, if, hypothetically, you have created one, then it might as well start fighting for resources against humans...

So, AI is bound to be limited in what it can do the more the decision belongs in the strategic or meta domain. And you very much want it to not grow a mind of its own, so that it doesn't write or approve code that optimizes AI's happiness, instead of yours.