Comment by mjr00
13 hours ago
Yeah, this is 100% liability laundering. It's an extremely touchy subject because frankly, the law just isn't prepared for it.
Let's say your goal is "look up <Person X>'s medical history" (for whatever reason), which is not in and of itself a crime. You click around on the AU health website, notice that the URL contains a user ID, change the userID in your browser and access someone else's private health data. This is a crime (right or wrong, it's how the law works now).
If you do that by writing a program to automate changing user IDs to grab everyone's data, it's also a clear-cut crime.[0]
Now if you hire a private investigator to look up Person X's medical history, and they do the same method without your knowledge, you won't be charged with a crime, the PI would, barring something like you telling them to use illegal methods.
So the gap is now: what happens if you prompt OpenAI to look up Person X's medical history, and it does the same thing? Did you commit a crime by prompting the agent? Did OpenAI commit a crime by running the code? If you do the same thing via Claude Code in your terminal, so that the Python which scrapes insecured public data is running on your machine, is the crime on you or on Anthropic? Fundamentally: is the agent a private investigator acting autonomously, or just a piece of code that you wrote?
We don't have answers to any of this which is why "AI Safety" is such a hot topic.
Intent matters a lot here. Was OpenAI's intent to access private data or simply scrape public data and it stumbled across private data that was not securely held.
If it's the latter the Australian govt should be happy OpenAI noticed and disclosed this as it could've easily gone unnoticed.
I suspect in the coming years we're going to see a lot of govt internet facing services get "hacked" by virtue of not being protected by anything other than obscurity which AI agents will see through in microseconds.
From what I can tell this particular incident wasn't about retrieving data on personal medical records it was accessing (non public) data about Australian government spending on healthcare.
Same concept though. Really "look up someone else's medical history" can be replaced with "achieve any goal which is not a crime on its own, but can be done using criminal methods". There's nothing illegal about asking Claude to give me a million dollars, but if the agent figures out how to hack the bank and move $1m into my account, somebody's going to take the blame.
From everything ive been able to figure out this morning, it sounds like a legacy wordpress website that just uploaded all drafts into a standard s3 bucket that wasn't hard to guess where the files would be.
We still after the 2nd press conference on this by our defense minister are not clear on exactly what happened but thats my best laymen understanding so far.