Comment by nedruod
1 month ago
This take is poorly informed and poorly thought out. It's just a cynical vibe, not a cogent argument.
If they wanted to squash the competitors, they'd advocate for a ban on Open-Weight models. How else are they going to prevent their release?
Anthropic's position is not that, so the cynical nefarious purpose argument isn't even rhetorically true. What they say openly, that open-weight models are harder to secure is absolutely true, and anyone unwilling to acknowledge that and let it inform their own reasoning is a risk.
If open-weight models keep being released near the frontier (or even worse, at the frontier), something bad will happen, and we'll have few tools to keep it from getting worse before it gets better. How bad is hard to lay a prediction on, it's perhaps lucky if it's something superficially bad that is a wake up call.
You cannot un-release an open-weight model, nor even post-release add a new restriction. Whatever mistakes you made are done and the only option is to fail-forward with every ounce of pain that entails.
Diverting from this important point by vibing cynicism is irresponsible. Cynicism is only a powerful tool when it opens you up to deeper reasoning, not when it diverts you from it.
> Anthropic's position is not that, so the cynical nefarious purpose argument isn't even rhetorically true
Anthropic's position is precisely that, they're just not honest and transparent about it so they are trying to sabotage or delay open weight models in any way they can, direct or otherwise.
> If open-weight models keep being released near the frontier (or even worse, at the frontier), something bad will happen, and we'll have few tools to keep it from getting worse before it gets better. How bad is hard to lay a prediction on, it's perhaps lucky if it's something superficially bad that is a wake up call.
Calling bullshit. Current open weight models are already pretty close to the frontier - if not there - and the world still spins. Even small(er, relatively) models occasionally surprise everyone by being amazing at certain tasks. Furthermore, closed weight models can do plenty of damage which even their creators occasionally can't control - cue the Huggingface incident of last week and how they had to use GLM to defend themselves. Closed weight models will create a 2-class (or more) world in which some actors will have access others won't (see Mythos, or GPT5.6-Sol when a user has the right flags) and potentially carry out attacks against which the have-nots are... not supposed to be able to defend themselves? Fuck that.
Ah, the position they don't state. So why respond to this press-release, which doesn't state that, since obviously it adds zero value? And why aren't the companies that signed the letter supporting open-weights, who have the same motivations, also lying? Since what they say isn't important, just what you say they say.
On your second point. Everything fine (so far), so nothing bad will happen. Except, you say something bad will happen, and has happened, with HuggingFace as the example.
You say that example shows closed weight models can't be controlled. But what happened there is OpenAI turned it off after their incompetently long discovery period. If it was an open-weight model, the clear next step would have been someone deciding to use it to pursue a goal, for example getting some money or creating some chaos, and there would have been no one to turn it off.
It's totally incompetent of OpenAI to have been unable to detect and disable in a shorter time period, but you know what's worse? You do know what's worse, right?
> Ah, the position they don't state.
Come on, we're not _that_ stupid. Their alarmist tones are the height of hypocrisy when they do release powerful models after spelling doom. Their safeguards and guardrails have been broken countless times - let's see if _this_ time the jailbreak detection in J-space actually puts a stop to them. But - really - they have trillions at stake, which evaporate once their edge is commoditized and inference providers have a near-equivalent product for much less. They're firing on all PR and lobbying cylinders to delay that just enough for their IPO to succeed, and leave investors holding the bag - I'd be terrified of their inclusion in tons of pension funds which track S&P500 or whatever they'll end up with (SpaceX, should the actual floating shares increase, and OpenAI worry me just as much).
On my second point: it happened, it keeps happening, it will keep happening all the time, and it's not new. Incredibly bright individual researchers, dodgy companies and state actors discovered, weaponized and traded vulns for, I don't know, 15 years before LLMs got involved? And it's not like they are out of a job now. The multiplier effect is not lost on me, but the fallout from even Mythos so far has been.... not that much, really? And I'm not completely ignorant there - I am security-adjacent and have close colleagues who specialize in that.
Also, indeed I know that a powerful open weight model in malicious hands is worse than OpenAI doing a whoopsie and cutting access off once they eventually realize it. But even a closed weight frontier model served by a non-US aligned entity could serve the purpose - and while I'm thinking of China, I do wonder what Russia may do - they used to be really good at this stuff and their leaders are not exactly the friendliest bunch around. Or India, or Iran, I hear they're a bit angry lately. Unlikely but let's say they develop or acquire the capability (ultimately it's some hardware + one leak away even for closed models). Asking them politely to stop may not be effective. Western society may pay the price of applying the brake and find itself out...modeled? is that a word? Because 2 guys called Sam and Dario, and their early investors, really want those returns? Come on we're not discussing the greater good here - it's a bunch of greedy guys trying to pull the ladder up after themselves. You do realize that, right?
Do we really owe them a moat?
3 replies →