Comment by john_strinlai
7 hours ago
>Felony Bench counts unique instances where AI agents inadvertently compromise or affect third-party entities.
a bit silly, as one typically has to prove intent (which is why security researchers don't get slapped with felonies all the time).
"inadvertently" and the existence of guardrails/sandboxes/etc make it pretty unconvincing that these incidents were intentionally malicious.
still a fun thing to track, but the name is just a bit overstated.
Under the law of Moses, if your bull gored someone, you were not responsible; but if it was known to be a gorer, you were responsible if you didn’t ensure it couldn’t gore someone.
I don’t know exact parallels in current law, but I presume there will be things like that.
The OpenAI/Hugging Face case sounded rather like OpenAI building a fence around their bull that was known to be a gorer, and then thumbing their nose at it and saying “nyaa! bet you can’t break the fence!” and walking away while listening to loud music.
In Australia, if you have a fire and leave it unattended and it escapes, it’s your fault, you were supposed to keep watching as long as it was burning.
Nobody got gored. HuggingFace may have the right to make demands; presumably they have already worked that out with OpenAI privately. Not really our business.
Nothing to see here. Just billion dollar companies producing hacking geniuses that are open for the public to jailbreak and use. This could never effect us, not really our business.
> I don’t know exact parallels in current law
You own a vicious dog, and it bites someone - you are responsible because you choose to own a dangerous dog.
A few claimed this might apply here: OpenAI knew their models are "dangerous", so they should be liable if they hack.
Can’t gross negligence or indifference to consequences lead to a felony?
Mens rea requirements are per crime and can vary wildly. Its difference between murder and manslaughter. The CFAA requires knowingly which is tough to prove.
i dont think any of these cases meet the bar of gross negligence, which is a pretty high bar. it requires proving a "conscious and reckless disregard".
which, again, sandboxes and guardrails and such would make a gross negligence argument unconvincing.
I think that if Hugging Face had filed a police report that OpenAI could have been charged with a crime.
I’m partially surprised that they didn’t do exactly that. If I ran a corporation I would assume any intrusion attempt by another company was intentional. Why wouldn’t I? Corporate espionage is super common.
I assume the answer is that these executives know each other personally.
5 replies →
How many escapes until it becomes reckless disregard?
6 replies →
AI corps rely on willful distortions of intent in laws to get away with moral crimes all the time.
Edit: changed labs to corps because it’s time to stop pretending these are places of science.
In the US, a felony by definition is any offense punishable by more than one year of prison (or by death) [0]. You could still call it silly on the grounds that AI agents aren’t put into prison as a punishment (though death might be considered an option).
[0] https://www.justice.gov/usao-ndil/programs/vwa-felony
Now that everyone knows this can and will happen, are any of the future incidents inadvertent?
What if the damage in future incidents is more than just "The LLM saw some stuff it shouldn't"?
What judicial system are you talking about? The fist incident in the list is something that happened in Australia. This is a technology used worldwide so I don't see how applying US standards works out here. Especially when there are countries out there that don't require intent and will look at the negligence presented.
>This is a technology used worldwide so I don't see how applying US standards works out here.
all three companies mentioned are headquartered in the usa, and im familiar with the CFAA in the us, so i am applying those standards. i should have noted that, sorry.
>will look at the negligence presented.
as far as i am aware, no evidence of criminal negligence has been brought to the public. has australia brought a case against openai or accused openai of acting negligently?
Roughly none of these fall under normal security researcher behaviors.
the mention of security researchers was to illustrate that intent is a crucial factor of CFAA cases.
"Doing crimes, but a robot didn't mean to and you don't know its intent" is understating the evil acts. Soon a robot can commit a murder but nothing will be done because of your line of reasoning.
> Soon a robot can commit a murder but nothing will be done because of your line of reasoning.
That's rather hyperbolic.
Are you seriously suggesting in that situation the robot should be accused of murder? The robot's operator could be accused of murder, but it could just be negligence without intent. Because that does, and should matter to the law.
Eh, if your take of this had any bearing to reality than I don't think most of the books written by Isaac Asimov would have gotten very far, but instead they've defined robot science fiction for decades.
In the real world we have no 3 ironclad laws of robotics. We are well aware that putting any sufficiently advanced antigenic system in a body that could be capable of committing a murder eventually will with the right set of prompts and environmental conditions. And these conditions likely have nothing do to with what we'd consider the human motivations for murder.
Hence at this point of time, any agentic robotic system that doesn't have safeguards to keep people distanced from humans is reckless endangerment.
1 reply →
it's not my line of reasoning, i didn't invent it. it's how the law currently works. intent is the crucial factor in CFAA cases.
maybe that changes down the road as a result of llm's and increasing frequency of similar cases. that has not happened yet.
its a meme not a metric
So is the comment you replied to.
"Inadvertent" from the perspective of the humans directing them. The intent behind the felony comes from the LLM agent itself. (No, I'm not interested in arguing with someone for the umpteenth time that LLMs can't have intent or agency)
with how the law is written today, software cannot be charged with a crime, so the only intent that matters in the criminal sense is the humans directing the llm.
You may not be interested in arguing but there are several blatant issues with the statement. If you're not charging the humans driving the software, who are you charging? The weights? The weights + the specific context window that produced the behavior?
I don't think this is a difficult question. The US has a history of civil product liability cases - see tobacco companies (Philip Morris), the Ford Pinto, the recent Meta cases, and the cases against character.ai.
From Investopedia [1], "[f]or a product liability claim to succeed, the plaintiffs in the suit must prove that a product was defective at the time it was transferred from the accused, and that the defect did cause the injury that's been claimed". It doesn't seem like a huge leap to me to argue that these models were defective insofar as they could not be safely used in a way that did not break the law.
I'm not a lawyer, and I'm not arguing that this is legally cut-and-dry, but I do expect that we'll have some answers about whether AI companies bear any sort of product liability sooner than later.
1 - https://www.investopedia.com/the-5-largest-u-s-product-liabi...
> No, I'm not interested in arguing with someone for the umpteenth time
... why my claim makes no rational sense.