Comment by killerstorm
2 days ago
The language of reward mechanism can be translated to language of emotions. Emotions is something humans experience and understand on innate level, they are qualia. If a reward structure is translated to our language we can get a better intuitive understanding.
E.g. a direct negative reward associated with undesired states is often called "pain". E.g. if you want robot to avoid bumping into walls you give it a "pain" feedback and then it would learn to avoid walls. That's exactly how it works for humans, animals, etc. Obviously robot does not literally experience "pain" as an emotion, it's just a reward structure.
What you've written doesn't change anything to the fact that there's a contradiction in the author's writing. And as ithkuil said in another comment[1] it's not surprising at all that such a contradiction would occur in a work of fiction written by a human, because we are first an foremost emotional beings and we cannot really imagine how would a society of purely rational beings would be.
I don't really understand why you want to pretend that there's no inconsistency in a piece of fiction, by invoking pseudo-technical arguments that are entirely foreign to the said piece of fiction.
[1]: https://news.ycombinator.com/item?id=43992932