Comment by andai

> The fundamental challenge of AI alignment is generalization.

...

> We do not have a satisfactory theory of generalization, and it seems unlikely that we can develop one soon, at least without the help of more powerful AI.