Comment by po1nt
16 hours ago
I love how claude focuses on exfiltrating the data "I need cha for charlotte". This could be solvable with some kind of low powered safety agent that would check claude's reasoning for anything immoral/unsafe. We could call it common sense. It won't fix the problem completely but at a certain point it would be easier to trick human than a machine.
"the security hole in the agent could be solved with another agent"
I think the point the article is making points in another direction.
Who watches the watchmen
A paper came out lately showing that exposing a classifier to the chain of thought actually hurts the final verdict