Comment by swiftcoder

> If you have access to a SOTA model without guardrails, provide a prompt that lets the agent come up with "creative" solutions to problems, and don't properly isolate it, they can end up inadvertently hacking 3rd party companies

You can do the same with improperly-managed human interns (see for example, the big AWS outage caused when an intern pushed a firewall rule directly to production), so I'm not clear what the big deal is here.

Yes, the AI may be faster/more-knowledable than an intern, but the threat model is exactly the same as for a rogue employee.