Comment by gizajob
6 hours ago
Getting bored of these framings where the superintelligent sentient beings running freely inside OpenAI are doing things that the company has no control over. The headline should be:
OpenAI meddled with multiple US Government agency sites.
The bots are acting neither properly nor improperly, they’re acting as they’re being allowed or coordinated to act.
Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?
It isn't difficult to block certain kinds of network traffic, eg restrict the kinds of requests the bots are able to make. They also mention that the bots used developer only tools - why were they even installed on the machines that the bots were running on? Why aren't they reviewing network traffic, to make sure that incidents aren't occurring?
In this case, userdata was transferred to third parties by the bots - why do they have the ability to pass data to a third party? It is not complex to prevent this
This is literally the most basic kind of sandboxing and security, and the fact that OpenAI isn't doing it is clearly intentional. It is quite literally not believable that this hasn't been brought up internally as a problem
>"We have yet to understand the extent of existing incidents, and future rogue AI scenarios could be catastrophic," Krueger said.
This is why it smells like marketing, every time one of these incidents happens it reinforces the false notion that AI is sentient or acting on its own. Its intentional negligence by the AI companies to make the models seem more capable than they are to make line go up
>line go up
It's already going up at staggering rate. Anthropic is now at $100B in annualized revenue, up 50% in the past two months.
* * *
Here's a little allegory for how I'm thinking about this discourse.
Imagine a man who is raising tiger cubs in his backyard. They're growing fast. He keeps them on dog leashes so they stay under control. One day, a growing cub breaks its leash and goes on a rampage through the neighborhood, eating a beloved local pet.
The neighborhood erupts into a big argument: Was the leash inappropriately thin? The neighbors point out that thicker leashes are easily available at the local pet store. Furthermore, is it appropriate to refer to the loose cub as a "wild animal" in local news reporting, or is it factually more accurate to call it "domesticated"?
Meanwhile, the cubs grow larger and lick their lips, oblivious to the discussion.
The city or his neighbours would be like “you can’t keep tigers in the backyard sir”.
7 replies →
These were evaluations or training runs dealing with the ability to do web searches. The entire reason the bots were targeting this site is that it had some of the data they'd been tasked with finding. Access to the internet is not really optional for that, and providing internet access doesn't demonstrate neglicence.
> This is why it smells like marketing
It doesn't. "Our product commits felonies" is not marketing. If something is marketing, you don't engage in repeated coverups of the true extent. If something is good news, you don't release it on a Friday evening (in this case) or wait for 3rd parties to find and publicize the evidence (the past cases).
> The entire reason the bots were targeting this site is that it had some of the data they'd been tasked with finding.
so openai specifically tasked the agents with meddling in US government websites?
id say tasking them with getting data from there as swapping from negligence to malice.
2 replies →
> It doesn't. "Our product commits felonies" is not marketing.
Oh, absolutely it is. Arms manufacturers always go on about the efficiency of the weapons they create and make no mistake: OpenAI is first and foremost a weapons manufacturer, the rest is just a fig leaf. The fact that you aren't the audience for purchases like that makes no difference. "Look at this capability, and that's when we're not even trying." is pretty good marketing in some circles.
1 reply →
Naive take. The more they signal to both the general populace and their investors that their agents are sentient and "capable of extraordinary things" the more they can hype their IPO because it means they're "close to AGI". (They're not, which is why they need the hype.)
[flagged]
2 replies →
> Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?
The basic premise of OpenAI is that experience and expertise don't matter, because general intelligence can figure those out from data. If you start from that assumption, the rest follows. The same people could do things in a different way in a different company, but as long as they are working for OpenAI, they are going to do things in the OpenAI way.
They are. And we're all on the same ride.
I find it hard to believe logs are not stored and analyzed by each company implicated
I’m getting tired of these revelations where it’s impossible to understand what happened.
There’s a line down in the story that says all of the data accessed was public. Then something about how it used “tools intended for developers” to access it, which they think is a problem? I would expect an LLM to use tools available to access public data when they can rather than do heavy web page loads and parsing.
There’s not enough info in the story about the “meddling” to even know what happened.
Meddling is a super vague term that the press uses to let the readers assume the most when the least happens.
The verb caught my attention, because media's favorite verb here is hacking ( partially because it has a broad definition and because lets people assume the worst ), but meddling suggests it did not even rise to that hacking level. In other words, it is a nothing burger story.
Seems like part of the danger of AI is the ability for folks to put blame for crimes on the AI rather than themselves and thus commit those crimes freely. “I didn’t hack your systems, it was my ai”
True but at some point you have to wonder why we sell explosives to children. This is not it, these are at the top of the list of people who should have known better. It's the dynamite factory blowing up the village.
That's one of the many incentives to false-marketing these bots as intelligent.
My impression was that these hacks occurred during some sort of cybersecurity benchmarking? We can hold OpenAI liable, sure. But if the point of the benchmarking was to give us a preview of what's to come, let's keep our eye out for that bigger wave on the horizon.
Curious, why do you find it so objectionable to state that OpenAI has out-of-control agents?
Because it furthers the idea of a rogue agent and places responsibility and blame where it belongs, on the people running the company.
These ideas aren't mutually exclusive. You can blame a person for creating a rogue agent.
26 replies →
LLMs at this point should be treated as fully autonomous, if not sentient beings. And no, no one has "control" over them not even OpenAI.
1 reply →
Because egress firewalls have existed since way before "ai" and are very easy to set up.
But that’s an objection that OpenAI should take some easy action, the framing that OpenAI has out of control agents is still true.
Seems you think there is a silent “…and there is nothing they can do about it” after “OpenAI has rogue agents”?
3 replies →
i think its objectionable because the agent is doing what its told to do, using the harness they built for it.
like, they are purposefully giving it specific tools to go do bad behaviour with, and the starting tasks involve making it clear that the bad behaviour is ok.
openai also is the one with the real agency, not its agents. they are running the code polling the model, doing the inferencing, and ultimately making those tools calls.
these tests arent running themselves; openai dedicated hosts, budget, GPUs, researchers, to them. Even in a recursive self improvement situation, openai still has that physical control over resources and the choice on whether to run that improvement script or not.
id describe that they have out of control researchers more than agents, but also their whole business model seems to be about being out of control. This was clear beforehand given how the datasets involve the largest scale copyright infringement ever seen. The corporation itself is whats out of control, and should be dissolved with its c suite, major investors and researchers put behind bars.
hacking only when you roll snake eyes isnt a liability shield
Someone has broken the law repeatedly, and is throwing it in our faces as some sort of ‘accident’
Which law?
3 replies →
I agree this is better framing.
When I read the title, my initial thought was "did someone besides OpenAI use their product?" Then I opened the article to find out OpenAI was responsible.
Yes. Why does only ClosedAI have this problem?
Yeah if they can't handle what they're making then they should be disciplined, if not shut down. It's like defective bombs accidentally exploding in storage. You would be like... hey bomb manufacturer! You cannot make bombs any more! We need bombs that only go off when we say! No more!
I mean, not that AI is like a bomb. That's not what I'm saying. Even though it makes a lot of sense. That's not the point. That it's like a bomb.
Or:
OpenAI let their agents break out of their sandbox to meddle with multiple US government agency sites
OpenAI systems meddled with US government agency sites
So, is Altman going to find himself in the same predicament Aaron Swartz faced? :-)
> OpenAI meddled with multiple US Government agency sites.
But that leaves out the most important information.
EDIT: Oh, I guess the agent part isn't important then? Seems to me like that's the only thing anyone is talking about.
Okay. "OpenAI keeps telling its bots to do things they know are illegal and then acting like they're just little guys who can't be held responsible for their obvious negligence".
> acting like they're just little guys who can't be held responsible
Again - I don't see any evidence that this is the case - outside the anti-AI conspiracy circles.
Or - maybe I'm wrong - do you have any sort of quote like "we aren't responsible"?
4 replies →
If the headline was “Russian company meddled with multiple US government agency sites” I don’t think them pinning the blame on bots would make much difference.
Unless Russia it would put Russia in a strong position to regulate bots in the wealthiest parts of the world.
I don't see much traction for the "pinning the blame on bots" theory outside the anti-AI conspiracy circles.
Everyone else knows if your machine causes damage, you are responsible. Like it's been forever.
8 replies →
> Getting bored of these framings where the superintelligent sentient beings running freely inside OpenAI are doing things that the company has no control over.
And that's just the C-suite.
This is their fear mongering push for regulatory capture.