Comment by DeusExMachina
1 day ago
> Perhaps you, a human, read this and think: Well, this world sounds kind of boring. Some of the machines think so, too.
> Most of the machines got bored of the project. But, all of a sudden, things began to get interesting.
> The result was like nothing the machines had ever seen. It was wonderful
> Machine society began obsessing over this development.
> The machines were impressed. And a bit scared.
Boredom, interest, wonder, obsession, being impressed and scared are all emotions that the machines in the story should not be able to experience.
Jürgen Schmidhuber introduced curiosity/boredom mechanisms as a way to improve learning in reinforcement learning environment:
https://people.idsia.ch/~juergen/curiositysab/curiositysab.h...
This mechanism can be formalized.
> Zero reinforcement should be given in case of perfect matches, high reinforcement should be given in case of `near-misses', and low reinforcement again should be given in case of strong mismatches. This corresponds to a notion from `esthetic information theory' which tries to explain the feeling of `beauty' by means of the quotient of `subjective complexity' and `subjective order' or the quotient of `unfamiliarity' and `familiarity' (measured in an information-theoretic manner).
This type of architecture is very similar to GAN which later became very successful
While this is interesting, gp's point still stand as the text explicitly says “There is no emotion” in the world of machines.
The language of reward mechanism can be translated to language of emotions. Emotions is something humans experience and understand on innate level, they are qualia. If a reward structure is translated to our language we can get a better intuitive understanding.
E.g. a direct negative reward associated with undesired states is often called "pain". E.g. if you want robot to avoid bumping into walls you give it a "pain" feedback and then it would learn to avoid walls. That's exactly how it works for humans, animals, etc. Obviously robot does not literally experience "pain" as an emotion, it's just a reward structure.
1 reply →
indeed, but ultimately this story is written by a human who while trying to imagine a world with "Just machines, bolts and screws, zeros and ones. There is no emotion. There is no art. There is only logic" cannot quite do so.
It's very hard to do so. It's so deeply wired in us. It's part of the mechanism of our brain. We appeared to be equipped with whatever it takes to feel existential dread and we feel whenever our thought wander to the possibility of humanity no longer being there. I hear people feel that when thinking about the heat death of the universe too.
Just see “bored” as a state of declining cycles of computation devoted to a subject. Obsessing as the opposite (or above some threshold).
Wonderful may describe “attention required, no danger, new knowledge”… etc you get the point. It's just written in a way that you puny human may get a "feel" for how we experience events. You cannot come close enough to our supreme intellect to understand our normal descriptions.
The superset of "emotion" is heuristic. Machines without heuristics wouldn't get very far. Their heuristics would probably look quite different from ours though.
I thought this too, but I imagine it's a bit of pathetic fallacy for the reader's benefit else it would be (ironically) quite a boring read for us from the machine's perspective.
[dead]