Comment by grey-area
6 hours ago
OpenAI are trying to create the perception that their agents are independent thinking machines because their valuation depends on the world believing this.
That is why ‘asked’ is important, and why we should question their framing of incidents like the hugging face hack too.
Enterprise customers don't like AI that can autonomously hack third party companies and cause felony-level cyberattacks.
IMHO, many enterprises are not worried about felonies or ethics, but rather the PnL, assignment of responsibility, and the bottom line.
Just have a look at various scandals of megacaps. Enterprises that seek to operate lawfully, responsibly, and ethically is the exception, not the norm.
(For example, look at Apple and how they were referred to the DoJ for criminal prosecution, over their execs and legal team lying under oath to a federal judge).
For most (if not all), it comes to a spreadsheet. And if you can do something with AI agents while diffusing responsibility for questionable illicit activity, that’s a selling point, not a downside.
The mental picture that enterprises like to play it safe is the PR imagery, not the reality.
Doesn't seem to have stopped them buying into it so far.
>OpenAI are trying to create the perception that their agents are independent thinking machines because their valuation depends on the world believing this.
Wouldn't the calls to slow the development of frontier AIs contradict this idea?
https://www.pacingthefrontier.com/
You aren't one of those "regulatory capture" people are you?
https://www.astralcodexten.com/p/ai-safety-is-not-a-corporat...
Because it takes some mental gymnastics to both believe
"it's all a stunt to convince investors their AI is/will be hyper-capable"
and
"it's all a stunt to convince the government to regulate AI so it won't be hyper-capable"
Convincing the government to regulate their competition while lobbying for laxer requirements for the big guys who have government contracts and get/give kick-backs isn't a far fetched scenario at all considering it's literally happened over and over and over and over again.
It is a far-fetched scenario given the tremendous number of eyeballs on the AI industry.
"...classic regulatory capture takes time, it’s a process of erosion rather than a battle, it happens in the shadows, in the backrooms, away from the public’s eye. As Culpepper argues in Quiet Politics and Business Power, business power goes down as political salience goes up. Regulatory capture and lobbying does a good job explaining why roasting coffee beans was defined as “domestic manufacturing”, thereby lowering Starbuck’s tax rate by 2%. It does less well at explaining big cross-industry issues the public cares about such as environmental regulation or race and gender discrimination regulation."
https://marginalrevolution.com/marginalrevolution/2026/09/wh...
A lot of these AI big shots were concerned about risk even before they started their current companies. Occam's Razor says they are expressing concern because they are still concerned.
4 replies →
Wait, are you trying to argue that regulatory capture isn't real, or that the federal government is trustworthy enough to itself to give monopoly control over AI development to?
Regulatory capture is a real phenomenon, but I highly doubt these incidents were faked in order to advance regulatory capture.
This type of elaborate conspiracy is improbable a priori, and it gets even less probable here, since it's such a silly strategy. For regulatory capture, you want to tell a story about how you're uniquely capable of delivering a safe product. Not a story about how "Woops we screwed up big time! But a less-regulated product from a smaller industry player saved the day!"
As OpenAI's chief scientist put it:
"Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world."
https://openai.com/index/an-alien-mind/
Announcing that "we have no idea WTF we are doing" is not exactly the move, if you want regulators to grant you privileged status. Realistically, it is more likely to e.g. create a harsher liability regime for you and your competitors, which is now being discussed by US senators.
As for the federal government trustworthiness point, that's a whole 'nother discussion. I'm usually quite pro-capitalism, but even I have to admit that this AI race is essentially a case of capitalism run amok. I wouldn't give the feds monopoly control though, I would tell the feds to shut it down, or something along those lines at least. This is an interesting site: https://pauseai.info/
6 replies →
Calling for a slowdown backs up the idea that these are extremely powerful and dangerous.
Assuming they're concerned with their massive pile of liabilities as well, asking for a government mandated slow down is asking what could likely be used as a force majeure.
>liabilities
"Every single senator seemed to think it was obvious we needed both much harsher liability regimes for AI developers and also new legislation, both very quickly. This was the complete consensus; the difference basically being degree."
https://thezvi.substack.com/p/the-ai-preference-cascade-reac...
Again, these galaxy-brained theories just don't make a lot of sense.
2 replies →
it's a stunt to convince the government that only they can be trusted to operate it and that their competition/small local deployments cannot.
Well that would explain why HuggingFace announced that the use of an open-weight model was critical to their investigation of the incident :-)
2 replies →
Imagine thinking that repeatedly demonstrating your inability to control your own models is "a stunt to convince the government that only they can be trusted to operate it"
Truly spectacular gymnastics. I'll give it an 8/10.
5 replies →
I don't think it takes mental gymnastics to believe that corporations want the government to believe one thing and investors another. You see the same thing in a number of other areas.
So I'm supposed to believe that the intent of publicly announcing e.g. the HuggingFace incident was to simultaneously persuade investors that the technology will rocket ahead, and also to persuade governments that it needs to be thwarted?
Why wouldn't it do the reverse? Persuade investors that the technology will be thwarted (and is thus a poor investment), while also persuading the government that the technology needs to rocket ahead with no regulation (to boost the economy)?
Most of these HN conspiracy theories seem like galaxy-brained just-so-stories to me.
OpenAI only confirmed the RubyGems incident when it was discovered by external researchers: https://www.reuters.com/legal/litigation/openai-agents-attac...
"@OpenAI this is the third time you've dumped these incidents late on a Friday. The first time this happened, I asked people around me if it might be intentional to burry it, and they assured me OpenAI had too much integrity to try to do that.
Perhaps one of your researchers can confirm that the unfortunate timing is a coincidence?"
https://x.com/SydneyVonArx/status/2108783466451984737
5 replies →