Comment by gewetensleegte

> But also, the agents absolutely went rogue.

They were also doing exactly what we know happens during unmonitored reinforcement learning.

The idea of RL is decades old, and that is the mechanism that went rogue.

The LLM made it more powerful and caused real world issues, but the literature is full of RL agents breaking physics sims, and doing completely out-of-the-box solutions if it gets them a higher score.

It just adds up to the labs setting up situations that should have been obvious how irresponsible they were.