Comment by deathanatos

2 years ago

I just can't read something about "safe superintelligence" and not hear the opening plot to Friendship is Optimal, a short story about My Little Pony (yes, you read that right), virtual worlds, and the singularity; it's a good read¹. One of the characters decides they can't make a violent video game, as the AI would end up violent. Better to build a game like MLP, so that the AI isn't going to destroy humanity, or something.

¹The story: https://news.ycombinator.com/item?id=36240053 ; you don't need to know or care about My Little Pony to read or enjoy the story.

To me, "safe superintelligence" presumes it is practically feasible and morally justifiable to create such a thing. I don't share those presuppositions.