Comment by Havoc
17 hours ago
That section about the agents trying to crack the PRNG is wild. Same for the heartbeat
Clearly not self-awareness per se but alarming line of reasoning anyway
17 hours ago
That section about the agents trying to crack the PRNG is wild. Same for the heartbeat
Clearly not self-awareness per se but alarming line of reasoning anyway
it's clearly incentivized by the RL rewards if you can cheat the task in a completely general way.
>"Clearly not self-awareness per se but alarming line of reasoning anyway"
Awareness is not necessary at all to create great harm. Biological viruses know nothing of what they do, yet destroy whole populations. I suspect the first truly damaging AI incidents will be similar; agent swarms locked into a self reinforcing reasoning loop that has no "intent" but is destructive nonetheless.
[flagged]