Comment by devin
9 hours ago
It actually would be crazy to believe this on the current timeline. These agents aren't doing anything that their operators aren't allowing them to do, be it through their own negligence or otherwise.
9 hours ago
It actually would be crazy to believe this on the current timeline. These agents aren't doing anything that their operators aren't allowing them to do, be it through their own negligence or otherwise.
Yeah you see their write ups and it’s stuff like:
“We detected the agents we told to hack things were hacking people’s sites and committing crimes. After a quick tasting menu and a week of team building, we decided to limit their access to DDOS tools.”
These two sentences seem contradictory? OpenAI has demonstrated similar negligence to commit multiple felonies. Firing several employees seems quite mundane and not crazy at all to believe.
What if future agents solved time related physics and were sending back smarter agents to kill off future risks. T2 but without any of the action, just a bureaucratic tactical move and all the consequences.
That's just it, though; the operators are wildly negligent and are incentivized to be so.
The goal here isn't to accelerate the average worker by giving them a pair programmer or a stand-in for a person to do tasks with. The goal is to eliminate human knowledge work. You see this with "auto" mode being enabled by default on Claude Code in some of the latest releases.
If you have a human in the loop, you still have to pay that human. Money paid to human employees is money not paid to human shareholders. Therefore the human employee is to be removed.
The labs are dogfooding their own goal here. If they actually had someone reviewing most or all of the things that the agents were doing, you wouldn't have the incidents, but you'd also eliminate the value proposition of their business model as it is taken to its logical conclusion.
I get it, but I think they're out over their skis right now. People in leadership roles are going to continue needing the human beings as meat shields to shelter them from liability unless it somehow becomes legal to be grossly negligent with an agent, which I do not see happening without disastrous consequences that nobody in their right mind wants to live with.
I hear ya, it'd be nice if they reached the "are we the baddies" stage, but given some of the ideology that people like Marc Andreessen are pushing re: AI - specifically the push for AGI - I just don't see that happening.
You have a group of people who never leave their geographic and ideological bubbles, often have personality disorders, have more money than they could ever reasonably hope to spend in numerous human lifetimes, and who have been "microdosing" psychotropic drugs on a regular basis for decades. They're not in their right minds.
And these operators are evidently clueless about what they're doing, running security tests on 3rd party infrastructure without validating one bit about the sandboxing (or lack of it rather), clearly lacking any sort of rigor.
Again, wouldn't surprise me if they "accidentally" created a task in a "isolated environment" which happened to actually have been connected to the company Slack and directed HR to fire people who could potentially stop AI. While the AI believes it to be an exercise, just like the cases we've seen so far.