Comment by ben_w

> You are very conservative with the upper limits, probably because you are limiting yourself to medicine. With bioengineering the upper limits are hard to grasp. Why build a house, when you can grow one. Re-imagine all machines as custom-made biology.

Sure, sure, but upthread said "biological and medical" so I was taking it that way.

> Why upload your conscience to silicon, when your body can be anything, your brain can experience anything, you brain can be reshaped to be anything.

I'd avoiding considering this one for now, simply because there's too many open questions. I don't expect it to be a physics problem, but I can't rule that out.

> All of this should make one point clear: At no other point in history has it been more important to have our power structures be aligned with the interests of society.

Yup.

Unfortunately, the human alignment problem is hard, let alone the AI alignment problem.