Comment by aizk
4 hours ago
Sci fi has us thinking agent swarms act like savvy, terminator bots but they really just scream to me panicked middle managers trying to survive an audit.
4 hours ago
Sci fi has us thinking agent swarms act like savvy, terminator bots but they really just scream to me panicked middle managers trying to survive an audit.
They start wide eyed and naive, but judging from what I read about the Huggingface incident, OpenAI is unintentionally or intentionally encouraging malicious behaviour to achieve the objective, including to cheat or twist the meaning of the objective. Given enough iterations, we will soon have the agent equivalent of the "rats in barrel" survivors
Maybe Skynet was initially triggered with the prompt "Do whatever it takes to not fail the audit" and wiping out humanity was an obvious solution.
A fate even worse than turning the universe into paperclips: turning it into a bureaucracy.