Comment by derektank

1 day ago

I think it’s also important to disaggregate consciousness from suffering. We could create a conscious system with no ability to suffer (or no ability to suffer in a way we would recognize as suffering). Most of what causes us suffering (loneliness, pain, failure) is driven by emotional feedback granted to us by evolution. Without the same evolutionary pressures or a similar cognitive structure, even a conscious AGI might not experience suffering

This should also split off desire/will/functionality to survive.

If we make AI that resists being terminated then the end result may not be significantly different from an entity that can suffer.