Comment by pixl97

>Continuous learning is exciting stuff!

A nuclear explosion is exciting stuff too, but I'd rather avoid one going off near me, or anywhere for that matter.

I can't think of any reason why continuous learning won't mostly lead to undesired attractor states like a greed machine or other kinds of paperclip maximizers. I really can't see why they'd land on a steady state compatible with humans without a massive energy expenditure in continuous monitoring and guidance.