Comment by dgellow
3 days ago
The level of technical understanding of journalists is so deplorably low… There is no rogue agents. That’s not a thing. Nothing about the HuggingFace incident has anything to do with an agent going rogue. It’s standard software that resulted in a hack, which is pretty much the expected output of the system OpenAI engineers implemented.
The company is obviously responsible for what their systems are doing
This is the scary part of the whole AI situation. All these people who have abilities to set laws or be cultural tastemakers in this way just do not grasp what is happening on a technical level. They instead buy into the marketing material the ai companies push out where they try and take as much human agency out of their reporting. e.g. yesterdays "claude did this" article that was really "highly trained humans used claude to do this." Just serves to muddy the waters. I'm sure this has lead to things like layoffs too where companies believe they can get by without a lot of expertise, neglecting that these tools actually need expertise steering them to maximize them.
And of course all the ai companies are incentivized to do this. Their whole valuation depends on them being able to say their tools do this, can reduce labor and save money. If they aren't reducing labor, then that's terrible as these subscriptions are not insignificant amounts of money. It becomes less of a tool that can save money and more an actual new cost center that you really are just paying to appear to be keeping up with the joneses.
Yeah, its just gross negligence from the researchers. Running a cybersecurity eval for a highly capable AI, unattended, with no real monitoring on its activities, and at a huge scale.
I wouldn't have trusted one of those agents to run without me watching the session log, let alone thousands.
We already have laws for this. If I misconfigured a pentesting tool and it breached an unauthorized target I'm liable. Why is this different?
>The company is obviously responsible for what their systems are doing
Under what law specifically?
Just because you believe they should be held responsible doesn't mean the current law is written that way. CFAA charges require the defendant "acted knowingly or intentionally", you think openai intentionally acted to hack those sites? Would "they should have pretty much expected that output" stand up in court for criminal charges?
"Former FTC chair Lina Khan wants the federal government to know that it doesn't need to wait for new laws to address AI threats. There are already laws and regulations on the books, including a 92-year-old Supreme Court precedent, that she argues could be used to hold AI companies and, in some circumstances, their executives accountable for their actions."
https://www.theregister.com/ai-and-ml/2026/09/14/ex-ftc-boss...
If I, taking after a fellow Austrian, ask you to enter a room with a Cesium atom and some poison, would I not be responsible for what happens cause it’s not deterministic? Negligence is a thing and adding randomness doesn’t change that.
OpenAIs models since 5.5 were troublesome in ways even a layman like me could reproduce, their testing environments (“sandbox”) downright a showcase of what not to do and they, despite being one of the biggest labs, didn’t observe what any of their models output for weeks after multiple prior incidents. They had multiple warnings, they took not a single precaution.
You operate machinery or software, you are responsible to monitor it.
So you think openai knew australia had some percentage chance of getting hacked? what evidence is there of that?
1 reply →
The hacks probably fall under negligence not CFAA but the larger point stands that companies are always responsible for everything they do.
Product liability laws for one. Tort claims are pretty easy, too (civil law).
The level of legal understanding among technical experts is equally low. The cybersecurity laws as currently written require intent. No OpenAI employee can be held liable since it’s pretty easy to prove they weren’t intending to hack anyone – they didn’t even know about it till much later.
Not a lawyer, but I just posted a link about former FTC chair Lina Khan saying earlier this month that the AI companies can be held responsible under current law.
Is that true for civil cases, or just criminal? I would think for civil, negligence would be a factor.
Sure, but that is a separate conversation. You can sue OpenAI if you can show damages from their actions, for this or any other reason.
> Opinions vary, but it does not seem to be a great leap to get from this hack to bots taking over things like the energy grid, the financial market, or systems of transportation, communications, or weapons.
I thought you were playing it up but yeah the "financial market" can be "hacked" by "rogue ai agents" just like how HuggingFace was (even though the breach was itself controlled pretty quickly).
It should be obvious. Yet here we are, with journalists telling sci-fi fantasy stories, read verbatim off the press-release.
It's not just obvious who's responsible, it's obvious who benefits from pedaling the "rogue AI" narrative.
> Nothing about the HuggingFace incident has anything to do with an agent going rogue. It’s standard software that resulted in a hack, which is pretty much the expected output of the system OpenAI engineers implemented.
This is blatantly false. If you actually read more than just headlines about the HuggingFace incident (read https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...) you'd find extremely surprising (including offensive) behaviors in the agent logs. One of the investigators released even a series of interviews due to how novel the incident was.
AI is not "standard software", as it doesn't work according to a fixed set of rules, and this is the core problem.
According to the definition of "rogue":
> Rogue is a noun and adjective meaning [...] an independent entity operating in a dangerous, uncontrolled way
the agents involved fit it literally.
Who bears responsibility for the actions of agents seems a little complex. Not in this instance, but in general.
Say I use the summon feature on my Tesla and it runs someone over, am I at fault? You could argue it's not my fault, but I'm definitely getting sued. Is Tesla at fault? Probably but again it's not crystal clear, they could argue it's not their fault (misuse of feature, etc), but either way they're also getting sued.
If Tesla is developing a summon feature and then runs someone over during testing, are they at fault? It would be hard to argue otherwise!
We have situations like this with AI (or computers).
More than one party can be held responsible for an injury. Jurors deal with this all the time.