Comment by mrweasel
5 hours ago
> Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.
It is still my belief that Altman wants one or ideally more governments to shut down or slow down OpenAI. OpenAI is going to need more cash to survive and Altman has run out of plausible lies. Having the AI breaks pulled by governments is basically the last chance to explain why they still aren't going to be profitable, and why they just need that next X billion dollars investment.
I don't for a second believe that an agent starts trying to hack backend system, when the form or API it has been asked to use isn't working.
> I don't for a second believe that an agent starts trying to hack backend system, when the form or API it has been asked to use isn't working.
I have seen coding agents on my own machine (in sandboxed VMs) start doing things while trying to accomplish what I've asked that I felt sort of exceeded my mandate (changing database passwords, poking at the egress proxy that's preventing them from accessing some domains). Not to the point of causing any real issues, but I don't have much trouble envisioning scenarios like this when using stronger instructions around pursuing the goal + a running in a misconfigured sandbox envrionment.
That said, there's a lot of potential upside for American AI labs if they're able to get people scared about AI, they can:
- To your point, claim the regulations slowed them down and paper over near/mid term financial concerns
- Get the government to create stupid regulations that don't actually slow them down at all, but do effectively lock out any future competition (and current global competition)
- Position themselves as the only organizations blessed by the government with the ability to make safe AI, therefore eventually allowing them to claim to be some flavor of "too big to fail" and worthy of a bailout, should the financials not work out.
- Effectively create a distraction that avoids further public conversation/accountability/regulation/liability re the more tangible sorts of problems their products cause right now.
> I don't for a second believe that an agent starts trying to hack backend system, when the form or API it has been asked to use isn't working.
Why not, isn’t that classic misaligned AI behaviour?
I can see someone coxing the agent into attempting into hacking a system, what I question is the agents doing it autonomously. The part I don't buy the agents trying an API, that's not working so it automatically switches to hacking in.
If the title had been "Attackers utilize OpenAI agents to hack Australian government website" that would be more believable. I'd also expect OpenAI to fight back and saying that their agents are being misused, but aren't inherently unsafe or autonomously break in systems. It's just that they don't. They openly speak of rouge agents, yet aren't sufficiently concerned to shutdown their services. OpenAI continues to speak about safety, yet they don't shutdown ChatGPT and Codex? How concerned are they really? It seems far more likely that they expect to benefit for having the public believe that their agents randomly hacks systems and "go rouge".
I'll never understand the "slow down AI" idea. Other countries simply won't slow down, why would they? Maybe a couple Western countries would but no one else will give a shit about that plea, nor should they.
Because you need an excuse to not deliver on you financial goals. It doesn't really matter what other countries are doing, if all you need is a reason to not IPO, because doing so would reveal that you've been deceiving investors for years and there's zero chance of them making their money back.