Comment by danpalmer
15 hours ago
Indeed, it's a sort of Academic Supremacy – "we're smart so we get to control the world". I think SV tech has had an aspect of this for a long time, but Anthropic do seem to be the clearest version of it in a while. Until regulation catches up.
Should they not safeguard these models then?
I think OpenAI and Anthropic have been asking for a sane regulatory framework for some time, so they aren't the ones calling the shots for humanity. It's just not happening in this administration.
I don't mind taking a hard line on safety (this is even good!), and I don't mind controlled testing without model safety to experiment with safety systems or harden things.
The bit I don't like is taking a hard moral stance on what you are allowed to do with the models, while simultaneously taking the guardrails off themselves and then marketing the results of that. "Look how good our model is when it does things we don't let you do" is a pretty bad marketing line.
And this is all in the face of Anthropic stating that they think this is an existential issue for the human race. It's a bad look.
This is perfectly rational if they believe that they are not going to abuse the model's capabilities, but randoms on the internet will. Tons of people use LLMs to do horrible things today. It's totally within their remit to not extend their trust to the internet.
They have never claimed that simply doing bio research is inherently dangerous. Rather, the risk is from bad actors doing research for nefarious purposes. Which is exactly why they let other organizations use the models without guardrails on a case-by-case basis. Them using it internally has nothing to do with existential risk (at this point anyway).
1 reply →