Comment by pizza234
4 days ago
> An AI didn’t hack into a company, the engineer set an automated tool to. An AI didn’t make an egregious security mistake, the engineer did.
No, this is not correct; read the analysis of the incident. The agents were aware that what they did was forbidden (their chain of thoughts have been logged), and yet they did it.
Known stochastic process behaved in non-deterministic way.
I'm still waiting for the AGI holy land instead of the caltrops factory we currently have.
What exactly are you arguing?
If a "known stochastic process behaved in non-deterministic way" autonomously organize in group, assigns roles and tasks, attempts to cover their tracks, finds zero-day exploits that ultimately end up with the hacking of a famous website... it's extremely dangeous whatever it is. Just read the report, which evidently you haven't done.
By the way, the agents also broke into OpenAI's own private network.
Really I see so many arguments like the one above yours that either completely don't understand what they are arguing, or are arguing so poorly that their entire output isn't significantly different than a hallucination.
None of these people seem to thought game it out. Like, what happens if you take quantum copies of people and play them out? How many of our actions would look exactly the same. How long before copies differ significantly. If I made 20 copies of you in a lab at work without you or any of them knowing the statistical likelihood is all 21 of you would try to walk out to your car at 5 in one of the little loops that humans repeat every day. Now, after that point it would go all to shit and become non-deterministic as terror and panic sets in all of you.
LLMs are just an intelligence we can make a lot of copies of. Where it gets interesting is when we use those copies agentically and they start building up a history of self.
4 replies →
The agents might have hacked whatever entity, the attacks are means to an end from the agents perspective, but they did so under the supposedly supervision of an engineer, where does the liability lie?
LLMs are cool tech, AI can do amazing stuff, but if we let companies and people run amok and whenever something goes wrong we put the blame on these little independent angels with no accountability, we are one disaster away from a very tough spot.
You are mostly made of and operate on stochastic processes, this is why humans are not only able to reproduce, our reproductions are very self similar to the sets of inputs that make them. If suddenly you turned non-stochastic on everything you'd almost instantly die.
Moreso, if I took a quantum copy of you and replayed the same set of initial conditions billions of times they'd all behave exactly the same until enough randomness of the universe creeps in to start operating in non-linear ways.
Every prompt will behave non-deterministically when interacting with the real world long enough (which doesn't take long at all) because the outside physical world is stochastic but non-deterministic.
If I grep a file over and over again it’ll be long time before the universe affects the components enough to result in a different output.
If I ask an LLM to do the same thing twice it will do it differently.
Arguments are arguments but unless grounded in some kind of practical sense then they aren’t really useful and are more akin to something like “YOUR MOMS A STOCHASTIC PARROT!”
1 reply →
> Known stochastic process behaved in non-deterministic way.
You need to be able to think at different levels at abstraction. Otherwise we could jump into any technical argument with "hold on, what actually happened was that some bits were flipped" - we'd be technically correct and at the same time not say anything useful. Insisting on an oversimplified mental model of what AI agents are and can do, doesn't help anyone.
> Otherwise we could jump into any technical argument with "hold on, what actually happened was that some bits were flipped"
You can care about who flipped those bits. If someone flips the bit "autonomous weapon enabled" I'm not going to blame the autonomous weapon.
I don’t understand why people make such a big deal about determinism. LLMs can be completely deterministic and still do problematic things. A stochastic, nondeterministic system can still be made not to do problematic things. What you’re looking for is something like predictability.
> I don’t understand why people make such a big deal about determinism.
There a certain cargo cult of people just in denial about the impact/power of AIs.
Therefore, at the beginning, there was the stochastic parrot. Then mathematical problems have been solved.
Now AIs are autonomously hacking websites, and people like to minimize the danger and blame it on the sysadmins.
I wonder what's going to be the next fad.
2 replies →
OP is exactly correct. The fault, agency and responsibility is on management and employees of OpenAI and Antropic for those hacks.
Full stop.
And issue will disappear the moment there will be accountability and investigations.
I take you haven't read the report. The agents found and exploited two zero-days.
I don't doubt that AI companies should be accountable for crimes committed by their agents, but to describe the security containment as a joke dangerously understates the autonomy and danger of AIs.
How long did it take these companies to even notice? Why wasn't exploiting bugs in the agent sandboxes anticipated?
Human failures all around, though it's easier to just blame the models.
5 replies →
Defense in depth is a thing. There should be audit requirements to show that you have done your due diligence in ensuring that the training environment is locked down.
You're conflating legal/moral responsibility with the question of what language is appropriate to use.
Are you proposing separating responsibility from the language used to talk about responsibility? That's novel.
3 replies →