Comment by barrrrald
6 hours ago
I have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous
In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it
You may disagree, you may think that they are delusional or have a God complex. Those are valid opinions. But I don’t believe that this is all a elaborate ruse for commercial gain.
I agree with this take. Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith. If this is an intentional media campaign it is a remarkably sloppy one. It has been effective because if you tell people they're going to die, they tend to pay attention - think of the grip the 2011 Harold Camping rapture prediction had on our collective psyche, or the 2012 apocalypse. But the messaging is inconsistent, the target audience is unclear, the stated goals are muddy, the whole thing is packaged in dense SF-speak, it's just a mess from a comms perspective. That doesn't suggest to me that this is a concerted effort to enable regulatory capture. Perhaps there are some cynics among the executives and the investors who are happy it's happening to the extent it brings about regulatory capture, but nobody seems to be pulling the strings.
> Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith.
The road to hell is paved with good intentions.
Dario Amodei is a 40-something dude who is obviously very, very intelligent. But intelligence is not wisdom and his "essay" here can easily be read as someone who opened a can of worms and doesn't know (or can't accept) that he won't be able to put the worms back in. But he's going to try because he believes he owes it to humanity to try.
It's hubris in its most basic form, even if the guy who has it looks nice and wears shawl-collar sweaters.
I see what you mean. I meant more that they genuinely believe the tech is dangerous, not that they are necessarily doing the right thing.
2 replies →
I think you're right, and that's worse.
This disaster of a PR strategy is not going to produce the outcomes Dario says he wants. It's going to make people hate him. Most importantly for his goals, it's going to make Chinese labs hate him--and they already hate him, because he treats them as basically terrorists. So if there is a "pacing", it won't include China, and thus may as well not happen.
That leaves only two conclusions, and really only one:
Dario believes what he says about safety, but does not understand politics and is prone to very counterproductive action, and therefore can't be trusted at the helm of a leading AI company.
or
Dario is lying about his goals, and therefore also can't be trusted.
Simplest explanation isn’t that they are attempting this well-documented, well-understood corporate tactic? Because believing AI doom is simpler?
God complex would be a pretty simple explanation.
Regulatory capture isn’t an “elaborate ruse.” For god’s sake, its Wikipedia page is 19 years old. I am not asking you to study political science or read Foucault.
If you are going to invoke Occam’s Razor, you can’t ignore the simplest explanation, which has a 19 year old entry on Wikipedia and over 100 years old historical precedence.
"For god’s sake, it’s Wikipedia page is 19 years old." Rich patronising from someone who didn't bother reading the wikipedia page of Its, the possessive form of the pronoun It.
I don't think so.
If they actually believed in it, then they would stop pushing the frontier of capability and instead focus on alignment, safety, and better tools for controlling/debugging AI. Then they would actually share their findings and tools.
They don't do this. Instead of reducing the competitive pressure and helping the industry to build safer more aligned models they are doing the exact opposite.
> If they actually believed in it, then they would stop pushing the frontier of capability and instead focus on alignment, safety, and better tools for controlling/debugging AI. Then they would actually share their findings and tools.
I don't think this is fair. They've published more about alignment and safety than any other AI lab. And in the absence of a coordinated pause, them pausing development unilaterally doesn't do much to advance AI safety.
I think they are genuine believers, but I think it's naive to not think that commercial pressure doesn't play a role in their positions, either explicitly or, perhaps more likely, subliminally.
The x-risk stance and commercial stance have evolved to be the same thing - Anthropic must win, and then everything else seems to work backwards from that. Can you believe it, the path they think is best for x-risk involves them becoming filthy rich. And threats to their commercial dominance like distillation get framed in a way to turn them into x-risk concerns.
Why not both?