Comment by atleastoptimal

Yes this seems most likely. Regulatory capture, essentially, if they can get legislation made around this. Also serves as a convenient way/reason to stop training new models ($$$) by calling for a slowdown.

Might be. But it's easy to reach the same result without it being about power.

They (like many, though probably not all, of the other main labs) think they have a better shot at getting their AI to be aligned with human values than anyone else does. Obviously they can't all be right about that, but they can all think that, just like most drivers think they're above average at driving.

So:

1. We make safer AI than anyone else; "safer" is a matter of degree even if nobody is perfect.

2. That includes all the open models.

3. Thus, the prudent policy option is to ban the use and dissemination of open source models

Waiting patiently for Anthropic's next month article, "LLMs made a fully functional intercontinental ballistic missile. We accidentally wiped Greece off the face of the Earth in the process. Tragically, we are not responsible."

You can make it very hard for companies to legally use open source LLMs but actually banning their distribution would be essentially impossible under the US legal framework.

Is it possible this post is the consequence of anthropic hiring people that are safety-minded?

I don't think the two authors were told about a super secret mission.

Problem, they haven’t shown that the labs can guarantee misaligned behavior either. And to this date have not shown a credible plan to do so.

Could we just outright ban any further work on AI? Literally force Anthropoc, OpenAI, Google/X/Metas group to all shutdown?

Every new story seems like opening montages of a dystopian sci-fi movie showing how we go to the dystopia bit-by-bit.

Anthropic and OpenAI's, at the very least. And it works! With all media spreading their points, they have a wildly successful FUD campaign. With all this talk about expecting major hacks in half a year, I'm seriously starting to think that they could attempt something very malicious and criminal like intentionally sabotaging their own pipeline to make a point, or even already preparing it. Before you scream conspiracy theory (which it is!), consider that it's a very small clique operating unfathomable amount of money, corrupt and insane/cult-like just enough. This is a terrible combination. They're hugely incentivized to pull something like this off and they could get away with it. People did a lot worse things for much less.

and, give them some kind of rights - based on “intelligence”, so that the model takes the blame and not the user, I hope it can be ruled very soon that the owner/driver/company is responsible, not the model.

This framing suggests that this is the sum of their business strategy, and strongly implies that it's being pursued cynically.

The risks posed by open access to highly capable AI are unprecedented.

The answer is not to ban open models; nor is it do nothing.

That we don't have answers technical political social or otherwise to how to navigate these risks, is not Anthropic's fault.

It's actively unhelpful IMO to amplify cynical takes which wave off or sideline the risk, or the reality that this is a very hard problem, among the hardest our societies have had to negotiate and navigate. It encourages a naive primary color tribalism that actively prevents process and progress.

I don't agree with point 2 on. They don't seem to want to ban open source or have much hope of doing so.

Seeing if models can do damage seems more like sensible safety research.

My worry about AI software doing bad things would be more the kind of people who make existing malware. The ransomware comes from criminals, not the open source community.

This is exactly what they're trying to do, and it will probably work.

Grok (MechaHitler) isn't an open model but it's plenty misaligned.

Yes, they are cynical accelerationists who basically want to go as close as possible to a machine god, destroyer of humanity, to force regulation that put them in charge. It’s pretty much what Amodei has been saying in his essays.

It’s a misanthropic death cult (and oh, surprise, all their predictions make them the sole possible saviors of the world! How surprising)

[dead]

I like how you apply open source to something that has nothing to do with software. The hard part of doing evolving AI is doing it the first time. Once one company develops the first sovereign AI, it will be in everybody's ability to build one in a short period of time.

AI will be classified as munitions (or a law that has similar restrictions) sooner than later. Having a products with the active capabilities to harm people is dangerous.