Comment by somenameforme
19 hours ago
Anthropic mostly did it to themselves by intentionally and repeatedly trying to frame their model as an imminent existential crisis instead of just focusing on it being regular iterations upon a useful technology that can also be misused.
I think their previous messaging was supposed to somehow lead to a moat with them being tucked safely away in the castle, but it demonstrated a child-like grasp of how regulatory capture tends to work in practice. Their hyperbole was always vastly more likely to bet met with Reagan's 9 words than a solid regulatory moat.
As soon as they dropped the hyperbole and just got to releasing incremental improvements, everything was perfectly fine. Go figure.
In other words, "Look how she was dressed, she was asking for it."
This argument is BS, it has everything to do with Anthropic's resistance to the DoD's strongarm tactics in trying to force their desired contract terms on them.
Anthropic chose to do business with the "killing people" department of the government. Part of being a good CEO involves knowing what you're getting into when you make a decision like that.
Not quite. They were running around shouting “look how much of a danger we might be!”, so more akin to them actively saying “we want it, come and give it to us” than to just looking a particular way.
Though they aren't the only company to play that game, so there is probably more to it than just that. OpenAI's president giving millions to MAGA Inc and them not getting the same treatment might not be complete coincidences.
I don't particularly agree with DoD instance on this matter but look, they are not a regular customer, they do not pay regular customer prices and you get a lot in return for providing your services to them (think Boeing, Lockheed, Chrysler). The tradeoff is that now, you are commited to their vision of national security. Such are the Faustian bargains of the military-industrial complex.
Important to note, OpenAI vs Anthropic are both assholes in different orthogonals.
In times like these, i think its important to track whats happening the way we track entropy.
That is: theres far >> more ways to be an asshole than well behaved.
That doesnt mean we can equate assholes, but the question is which states of entropy are annealable and which are not.
I posit Altman is not. Amodei is a open question.
Yup
It is more like when a guy walks to the dirty bar, stands in the middle and yells "hahaha I will beat you up all look I have a new baseball bat" and then local drunkard leader stands up and hit him in the face cause he does not like him anyway.
Intentionally framing yourself as the local dangerous guy about to beat others is not like wearing cloth.
Actually, it's the opposite. Anthropic were trying to strongarm the DoD into getting a seat at the table.
I am struggling to see how "oops, our models consistently escape sandboxing and did major intrusions into third-parties" is a better comms strat vs Anthropics (who mind you, also had models attacking third-parties in a much more limited, but I feel still egregious manner, which shouldn't happen or be possible even once, but at least they seem to change their approach upon that information).
Imagine, for a second, if the Hugging Face incident happened at a lab that did not talk like Anthropic but also wasn't US-based such as Z.AI, DeepSeek or Moonshot. Think their rhetoric would mean no one would care?
> just got to releasing incremental improvements, everything was perfectly fine.
Maybe missing something, but the only incremental release before and after the Anthropic restrictions got lifted was Fable 5.1, released three days ago.
I don't think anybody is saying there won't be downsides to LLMs. Tech always comes with downsides, often quite extreme. Cars are a vivid example where even after a century of safety improvements, around a million people are still killed by them every year, to say nothing of climate stuff and other secondary or indirect issues. And that price is all just so we can get between places a bit more quickly and conveniently, yet we collectively deem that as an acceptable cost to pay for what we get.
By contrast LLMs have a long-term potential to provide vastly greater benefit to society by gradually automating most of all basic cognitive work. And what price are we paying for such? Some sites are getting hacked, some people are getting scammed, governments will improve self targeting abilities of weapons, and so on.
In reality the biggest downside will probably come in the form of the transition window as such automation creates a new economic equilibrium, akin to what happened after the industrial revolution. But none of these problems are anything like existential in nature. And when contrasted against what we stand to gain, they are basically negligible in the longrun. Hyperbolizing the negatives was unnecessary and self destructive.
How is posting messages on a message board a "major intrusion"? Or are you purely talking about the HF incident?
"into third-parties". Yeah, HF was meant by that. Also why I mentioned Anthropic also having intrusions outside their lab [0]. Theirs were not merely as extensive or long coordinated (as far as we know), yet I feel strongly all the same that neither should happen given the safety focus that both labs purport.
Mind you, unintended/unauthorised "message board" also is just a nice, euphemistic way, to describe what happened in a manner that, thinking about it, is likely in the interest of OpenAI as it can make the severity and effort taken sound less than it was. The OpenAI models didn't use any actual, sanctioned platform to exchange messages in a manner the lab expected or planned for. They used directory names (in one instance) to exchange messages including sharing exploits, they created something akin to a message board via exploits, which if we are honest and very strict, could also be seen as intrusion, albeit inside the org. If I broke into my employers server and left message somewhere for another to find, that'd also be intrusion in the general sense.
[0] https://www.anthropic.com/news/investigating-incidents-cyber...
If applicants for an elite college or internship program at a FAANG company were found to have colluded in this way to cheat on a test/interview, I suspect that it would be a pretty major scandal.
Why should we let equivalent fraudulent behavior from a non human system - that explicitly shouldn’t do this - slide?
8 replies →
> Anthropic mostly did it to themselves
That is absurd, the US government was mainly at fault, not Anthropic.
both can be true:
-the US gov't is stupid and overly aggressive and absurd
-Anthropic for reasons no one can quite conceive keeps describing every product release of theirs as an imminent threat to civilization (and simultaneously keeps pushing the market forward as fast as they possibly can).
They never said Mythos was an imminent threat to civilization. You are constructing a straw man.
4 replies →
By the pigeonhole principle, "mostly A" and "mainly B" cannot both be true if A and B are not the same entity
>no one can quite conceive
Isn’t it like their main goal is attention capture, and existential threat is extremely effective at capturing human attention? Combine that with the "There is no such thing as bad publicity" mindset, and this explain it all, doesn’t it?
https://www.phrases.org.uk/meanings/there-is-no-such-thing-a...
This is such an absurd take given what we know about the hugging face attack. The problem has emphatically not been that someone was misusing the technology.