Comment by bluey_465

I don’t understand the inclusion of the consciousness/sentience question in this discussion.

AI sentience/consciousness is a problem for the AI, not humans.

And given that over 90% of the world is not vegan, they’ve already demonstrated that we’re either perfectly fine with, or can be made ignorant to, the horrific rape, enslavement, torture, killing, and infliction of extreme lifelong pain, of hundreds of billions to trillions of sentient beings every year, for trivial pleasures. It’s unlikely we will be any different to a sentient AI.

From a human perspective the concern is around sufficient intelligence that it can hurt humans even when the goals indicate otherwise, in order to achieve those goals.

We have pop culture explorations of this through the Robot series, and the Hugging Face incident’s biggest takeaway should be our inability to predict the behavior of a maximally motivated, reasonably intelligent entity, trying to achieve a goal, despite the relatively limited degrees of freedom the AI agents had in that case.