Comment by serf

>If you have access to a SOTA model without guardrails, provide a prompt that lets the agent come up with "creative" solutions to problems, and don't properly isolate it, they can end up inadvertently hacking 3rd party companies.

it's pure delusion to think that's a SOTA specific quirk. DS/GLM/K3/Qwen/Claude/GPT/Gemini/Grok will all break CFAA laws with clever prompting, and they'll do it well if given the harness and tools they need.

This is evidenced by a huge uptick in game hacks and reverse engineering articles, some even featured on this site.

the reality is that it doesn't take a superintelligence to do something against ' the law ' , and 'being hacked' varies from victim to victim.

Will Phillips consider themselves hacked when a clever user prompts an AI into getting their toothbrushes to dump rom? Is it 'hacked' to clean-room re-implement a video game net protocol in order to produce private servers?

Judges opinions vary.