Comment by NumberWangMan

Heh, forgive me if I find the metaphor of “let’s send all of humanity into a black hole because it might be great” less than comforting.

I think the most likely scenario where we make something much smarter than us is bad. I don’t agree with all of what Yudkowsky says, but “Nature is allowed to kill you” is something that I think not a lot of people grasp on a real intuitive level. We are totally allowed to invent technology that gets out of control and kills us all. It could have happened with climate change if the earth had stronger feedback. We’ve been lucky so far, but as humanity gains power, we need a corresponding increase in our ability to coordinate and be careful to match.