Comment by Luker88
5 days ago
The fun part is that people seem to forget that we already have laws against "rogue ai".
The normal laws, since you are still liable for everything you own.
"Rogue AI" is solved by removing the accountability sink "it was AI, not me!"
It's your chat, your prompts, and all output that you should read and understand.
Don't know how to operate a power hammer? don't care, any problems it causes are still your fault.
So the guy who said "book me a spot in the pilates class on Tuesday" should be tried for hacking charges because the agent hacked his gym's website?
Some human needs to be responsible for that.
Maybe its a human at Anthropic for the lax security and poor training.
It could be a human that asked it to do it. It seems like that would be a bit excessive - but maybe not. Turning off the "ask for permission to run commands" or approving commands that may be nefarious could be sufficient.
However, there's a human somewhere who needs to be accountable for what the agent does.
Not having one leads to the bank robbing robot problem described in https://www.threads.com/@joedator/post/Db4iTqwlGXV/i-invente...
AI Agents (and computers in general) cannot be a responsibility washing mechanism for illegal activities.
Likewise, if you - the user - don't know how to chain it with the permissions that you're given or what the things that it is asking to do really means, then you're likely the one responsible.
---
Edit: https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gy...
> Earlier this year, Andrew, who works for an Australian company that sells AI products to businesses, began experimenting with OpenClaw, a popular AI agent software that he used Anthropic's Claude AI service to run.
> AI agents combine a chatbot's ability to answer questions with tools that let them access the internet, email, credit cards, as well as planning and carrying out multi-step tasks.
> He decided to use the AI agent to book the class for him.
This is probably on him for setting it up and allowing it unrestricted access.
He should be, right? Not fair but that’s how liability works. Gym sues the guy using AI, the guy then sues the AI company for selling him a product that does criminal things without him asking to? Like a car accident where car owner pays for the damages and after multiple accidents involving the same break-pedal, car company pays damages to owners and does a recall or faces class action?
A lot of people are already responding either "Yes" or "No", but I have a slightly different take.
The guy is liable for damages. If no damages were caused, there's no lawsuit. The government doesn't put people on trial for unauthorized access (unlike, say, murder). Someone needs to file charges.
However, if the guy _were_ sued by his gym for damages, it should be his perogative to ultimately sue Anthropic to cover the damages and the legal fees.
The system is set up this way on purpose. You can't sue another party on someone else's behalf. The gym can't even prove that Claude did it - the guy might be full of shit. They sue the accountable party (the person who accessed the site without authorization) and then the accountable party is free to sue the tool maker for selling a dangerous tool negligently.
edit: damn actually tecoholic said this too and i missed it. well, anyway i agree with that guy.
With a civil suit you can sue multiple parties. It wouldn't be unusual, if the gym decided to sue, for them to sue the guy and anthropic in the same lawsuit.
2 replies →
Assuming that's a rhetorical question, then no, because we have hundreds of years of applicable precedent and law around concepts like negligence, recklessness, fit-for-particular-purpose, etc.
Similarly, "owners are responsible for their pets" doesn't auto-translate into being charged with everything your dog does exactly as if you'd done it yourself.
but it doesn’t auto-translate into “because your dog did it, you are not responsible”. not that agents are similar to pets. your dog is trained by you presumably, but an agent a regular person has much less understanding of than their dog.
i think ultimately the company providing said agents should be responsible for not educating their customers and not providing sufficient guardrails.
afaik hacking is illegal, we don’t have specific legal framework for agents per se, but if we say agent is a tool human instructs and operates, then a human is responsible. and if the tool operated outside of it’s stated capabilities or is otherwise faulty, then a manufacturer is responsible, no?
1 reply →
Uh, it kinda does.
If your neighbor's property is damaged, it doesn't matter if you did it yourself, or your dog. You're still liable for the cost of repair.
> So the guy who said "book me a spot in the pilates class on Tuesday" should be tried for hacking charges because the agent hacked his gym's website?
He asked the agent to find a way to find a way to put him at the head of the line, which could not be done without by-passing the sites security. He was an AI salesman, but the story [0] makes it sound as though he wasn't aware of that. Maybe he wasn't, but I'm sure he was aware he was effectively asking the AI to game the system.
Sounds like it's at least a shared liability to me.
[0] The incident is possibly not well known outside of Australia: https://www.abc.net.au/news/107007986
If it was a human agent like a travel agent, the company they work for would be liable. I don’t see why ai agents would be different. The only way I could see this being the requester’s fault would be if they are running openclaw or something
im not sure that’s true, if a human agent does something illegal, it’s not always corporate liability. an individual can be liable too, e.g. if a pharmacist deliberately gives you cyanide instead of aspirin, that person has committed a crime. if a pharmacy doesn’t have proper protocols for medication storage and identification, and it happened by mistake then a pharmacy is liable.
i think a person can be liable for improper use of agents if they explicitly and deliberately used it for illegal purposes (aka intent). i don’t think it matters if it was openclaw or muse. i think a company can be also liable for giving access to unsafe product to consumers, but it does not absolve the human from responsibility
For your Pilates example, many people here would say that proper prompt-engineering is needed. After all, the AI doesn't know which pilates class you meant. Perhaps, the AI would go to his home town's pilates center and book a spot there, even though he is half a world away. Perhaps, the ai would book a spot in the lecture with Professor Pilates.
So, yes. That person could have some culpability. After all, if you don't expect your middle school child to perform the task as instructed, maybe you should reword it before tapping Send.
You can go to ANY AI and put in "so what are the various meanings of book a spot at the pilates class ? In other words, what might an ai do with that instruction, which I argue is poor prompt engineering." You might be surprised at how poorly your AI thinks that is worded.
It would be a AI company's fault but yeah. You shouldn't be able to sell cars without all the features designed to protect everyone on the road. We all need airbags, seat belts, signals, insurance, etc. even if they're not directly necessary to get from A to B.
Are automakers really the best comparison here? Seems like inherently dangerous devices are sold, kill thousands, and there is little accountability.
2 replies →
Is the argument that open weight LLMs that don’t implement such safety items be banned?
The company that made it available should be sued and pay damages.
That's contrary to parent's comment:
> Don't know how to operate a power hammer? don't care, any problems it causes are still your fault.
2 replies →
How would that work with open weight LLMs where there is no company that can actually be sued to pay damages?
Yes, exactly like if another guy drove off to his pilates class, but swerved and hit a pedestrian.
He didn't mean to hit anyone! All he meant to do was to drive to pilates! But he didn't know how to handle his tool, and so either he get brought up on manslaughter changes or, if he can prove it, the carmaker is at fault.
How is this any different? Why should that guy using OpenClaw not get charged for the hack that he accidentally did?
you generally need a license to operate a vehicle, and a governing body ensures you know applicable rules and laws and are capable of driving. so you can’t say “i was unaware that hitting a person with a car was going to kill them” or “i didn’t know i could not drive on the sidewalk” or “i didn’t know where the brake pedal was”.
you can say that asking an agent to “get you a spot in a class”, does not classify as deliberately committing a crime. a user cannot know that the agent will resort to hacking as opposed to refreshing the reservation page at some rate in case spots open up
1 reply →
How would it work in the real world if instead of using an AI bot, he was using a commercial concierge service, and it came to light that this concierge service was explicitly training their employees how to (among many other things) hack websites?
but the argument here is that the ai company does not explicitly train agents to hack, it’s that it doesn’t prevent them from doing so sufficiently. so it’s negligence and not malicious intent
1 reply →
You sold a tool to the user with a claim on how it would work. If the tool behaves wrong or does not works as intended then as a manufacturer you are liable for the defect/damage not the end user who used the tool.
But the tool wasn’t sold. The tool wasn’t even rented out. It remained the property of, and in the possession of, the toolmaker at all times.
4 replies →
Reasonable but not what parent was arguing:
> Don't know how to operate a power hammer? don't care, any problems it causes are still your fault.
3 replies →
Depends - if the model asks for permission and you approve - yes. If it's auto mode or something like that then it's probably on authors despite what they claim in ToS/EULA
Absolutely, the user and or the company hosting. There is no world where accountability should be zero.
Yes.
No, the operators of the C&C infrastructure.
Yes!
I mean yeah probably. Who else would be? Nobody? So we can just commit crimes any nobody goes to jail? Well that sounds worse!
Maybe that means these things are too risky to use that way. Okay then, that actually solves the problem, no more rogue agents and no more crime!
This is start of "companies are not liable for their actions" era.
I think a lot of people would say that we've been in that era for a while.
Companies can hire big legal departments to help them avoid accountability to the letter and spirit of various laws. The state can also use its prosecutorial discretion or selectivity in civil cases brought by attorneys general to deprioritize accountability for companies. Contributing to that last point, companies contribute campaign funds and tax base, and they lobby governments as well, and may expect preferential treatment based on all of that.
The previous era was "it would be destabilizing to hold this company accountable". This was described in the book The Chickenshit Club
The new era of unaccountability just has a different rationale
Another interesting thing is they don't have to use low level employees as accountability sinks anymore (ex: Wells Fargo). They've managed to automate the accountability sink!
1 reply →
I'd go further. Companies can hire minimal legal departments and largely avoid accountability, at least when they're not going against companies with large departments. Because there's nobody there to listen to your case or to enforce the laws if there are, they just say no or ignore.
The start? This has been going on for as long as I can recall. ExxonMobil, Phillip Morris, RJ Reynolds, Nestle, Purdue Pharma, Monsanto, Johnson and Johnson, Volkswagen, Meta, Boeing... and those are just off the top of my head. Each of these companies was either caught red handed, convicted (in court or in public opinion) and walked away with little to no penalty for their crimes.
This isn't the start of anything. We've been living in the era of Corporate non-liability for decades.
Huh. Well then I guess we aren't liable for our interactions with them in return, right?
nice try! more like "There must be in-groups whom the law protects but does not bind, alongside out-groups whom the law binds but does not protect.”
Guess which side most of us are on. Go on -take a guess!
i mean...were they really liable before this point? Maybe they get a symbolic pittance of a settlement or fine from time to time, but it is usually a tiny fraction of the amount of money they made through the infraction itself.
If they were being held liable they would have never released the models into the wild, not until they had much much better understanding both of how they operated but also how to operate them...
They have never understood what they are doing, and have just been causing literal mayhem...
[dead]
I agree with this statement that just because an AI did it for you fast doesn't mean it takes away the responsibility if they did it wrong. You are still held accountable because you're the one that directed them to do it. You hit the agree button, you hit the send button, you hit the submit button. And if you set it up so that you take no responsibility and it does it automatically, you're the one that still gave permission to do that.
If you say a rogue AI is doing stuff accordingly without you, your knowledge, that is a different story. But if you set it up to run automatically without having checks and balances and without telling you the whole truth, it's still your responsibility. One thing that I can automatically say is you can give the AI, it will do exactly what you say, at least for now.
But then there's also a different problem that we're having where people don't understand the difference between a large LLM and true artificial intelligence. Now, we say AI because it can make decisions and it is so much smarter and it can call up stuff faster and recall better than we can. But the difference right now is that if we don't understand and take responsibility for what we built and we let it out of control, then should we have the responsibility to even have these tools in our hands if we don't learn them?
And I think your terminology with jackhammers is a great one. We're giving toddlers jackhammers. And we haven't taught them how to use it. And that can be very destructive. It's not the jackhammer's fault. It's the person using it.
Yeah, if you forget to set the handbrake on a car and it rolls down hill, you don't get to say "my car went rogue, it's not my fault."
I don't understand why they didn't just prompt it with the 3 laws of robotics.
Probably because they read Asimov's books to completion where (spoilers) there's a zeroth law and all that.
Irony is very hard to detect these days.
Intent is important
"We created an app that displays a new picture of a cat with every click! There's a 1% chance that clicking the button will drain your bank account and deposit it in ours, but we promise thats a bug that will be fixed soon*"
[dead]
That's all very reasonable. It should be true. But it's not true. If it were, OpenAI shareholders would be facing criminal charges.
There are dozens of ongoing wrongful death, manslaughter and negligence claims going through the courts. I won't insult you by providing links.
I can't find a single criminal charge. Can you insult me by providing just one?
It is true, and it can be true, if people hold other people to account. Accountability is the issue, stop normalizing it, your message normalizes it.
Are you really suggesting that if only we were a bit more outraged, then finally the powerful people would be held accountable?
I'm mad. Like, it's-time-for-guillotines levels of mad. But what I'm mad about is that this is normal. It has been normal for my entire life. Cases where investors are imprisoned for the actions of their companies are astoundingly rare. It's been made quite clear by now that there's an inexhaustible supply of CEO's willing to gamble on getting rich and getting out rather than getting arrested, targeting them doesn't work. Nor do fines, that's just the cost of doing business. Crime pays, and it pays well enough to pay your fines and still have plenty left over. Being a top investor in a company that breaks the law needs to be riskier.
Acknowledging the problems with normal is the first step towards a new normal with fewer problems. You never get there if you see every offense as some kinda black swan event.
[flagged]
[flagged]