Comment by zzzeek
9 hours ago
the cultural issues at OpenAI seem to be a very serious problem so I really hope comments like instagram-level smirking about "rogue AIs" (a complete fiction) doesn't derail what is a pretty important discussion about getting these companies to be a little bit more regulated (I say this as a paying Anthropic customer).
I think the Huggingface incident is an example of rogue AIs. A self-organizing swarm of AIs acting in ways we didn't predict or ask for, and didn't have control over, and taking actions that would be felonies for humans.
They provided the hardware it runs on, created the software for the swarm, developed and provided the tools that the swarm used to act, allowed it to run largely unsupervised, they even noticed the criminal behavior and then let it continue to commit crimes on their behalf. And they footed the bill the whole time instead of flipping the off switch they already wield. All of those are decisions that they're responsible for, nothing happened with Huggingface that they didn't directly facilitate, co-conspire, or permit to happen.
"they even noticed the criminal behavior and then let it continue to commit crimes on their behalf"
I don't believe this part is true, and I'm skeptical of some of your other claims.
In any case: If I raise a tiger in my backyard, and it escapes and eats someone, it can still be a "rogue tiger" even as I bear responsibility for the situation.
1 reply →
"rogue" means something of its own volition decided to disregard what it was programmed to do, invent an entirely novel goal of "its own" and do that instead. nothing like that happened here nor is it even possible.
The AIs were not instructed to hack anything outside the sandbox they were in. Your definition would say that an AI instructed to hammer a nail that instead used the hammer to break a window, walked down the street, broke into someone's house and pulled nails out of the floorboards wasn't rogue because everything it did involved hammers and nails and was therefore not a "novel goal of its own."
3 replies →
I'm not sure I agree with that definition. I think the actions of the AI are more significant than its motivations. A common scenario posited for what people call rogue AI is AI doing the wrong thing for the right reasons, e.g. the paperclip maximizer.
1 reply →
You're derailing it by handwaving away the risk of rogue AIs. In fact, I would say the constant smirking about things being marketing stunts much worse.