Comment by tosh

8 hours ago

the way to avoid these problems is not to hope for the user or the agent never to make mistakes

it's designing the environment and invariants so whole categories of failures can not happen at all

the agent ui nagging the user for approval is a ux anti-pattern, we already know how well this works for operating system permission dialogues