Comment by NitpickLawyer
15 hours ago
> So they admit that they could do more, but they won't?
Yeah, there's another paragraph:
> Identifying and removing cyber-related data. During development, we observed vulnerability research performance improving steadily as post-training scaled. Tracing the source of this trend, we found that cyber-related content was embedded within the general mid-training corpus, and cyber-related tasks were present in the RL environment mix — a natural byproduct of training on broad data. In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness. We will continue to apply appropriate controls at every training stage, of which cyber-related data identification and restriction is one example.
(emphasis mine)
It's clear that cyber is a touchy subject right now. But it's also clear that RL works on it, because it's highly verifiable. So smaller models are going to become very interesting, because they can be fine-tuned later to cover this field.
I'm betting this is related to China's Announcement No. 48 last week about AI export controls, since releasing an open-weight model is effectively export: https://www.jdsupra.com/legalnews/china-proposes-new-export-...
> Prohibited: high-performance detection technologies used to counter cross-border data transmission security risks, including deep-synthesis forgery detection (266401J); AI cyber offensive capability technologies, such as malicious code generation and mutation, automated mass generation and distributed large-scale deployment of anti-detection variants (266402J); large models trained using China’s Core Data (266501J); and high-performance AI agents designed for highly sensitive scenarios, such as critical information infrastructure, national defense and core government affairs (266502J).
And that's why we need models that understand "cyber" or application and data security. The offensive models are powerful, and they are in the hands of script kiddies now. The models writing our software and designing our cloud infrastructure had better understand these issues as well, just to avoid coding in vulnerabilities.
They also show scores for a "Kimi model w/o cyber risk-reduction" https://statics.kimi.ai/kimi-blogs/kimi-k3-1/20261023/xdev-b... Presumably State Security will get access to that on request.
I think this is misguided. Considering how much code is AI written these days do we really want LLMs writing insecure code because we lobotomised them?
I think it's political. Certain companies are exploiting as much as possible to push the boogeyman of AI for regulation.
It's exactly this.
If the "scary Chinese open weights models" can do "dangerous cyber", then we might see regulations spring up against open weights sooner than later.
I hate Anthropic and OpenAI. As companies and leadership, anyway.
Dario and Sam are 100% to blame for this. They're trying to turn us all into jobless slaves, sex workers, and Squid Game pawns.
The only good future is open weights. They're trying their damnedest to keep us under boot.
I hope the Chinese companies (and I say this as a patriotic American) slay these two companies and destroy their moats.
In my mind, the two most dangerous people in the world are Dario and Sam.
We need open weights to win.
14 replies →
Anthropic: Touch too much security stuff and we ban you, and keep your money.
Kimi: We took out the worst of it so have at it for free.
Everyone else: Back to my ablated model on openrouter for 1/10th the price where I can brute force issues.
Ablation only works to remove stuff added in post training, if the training data does not contain the material then ablation will do nothing.
Which ablated model do you recommend for this?
1 reply →
It’s not much of a deterrent to a motivated organization.
They are basically saying that models themselves are already capable. If you fine tune them with some cyber security tasks, that should make them potent. :-/
There is no deterrent to a motivated and well resources organization. Post training a model in general, let alone just for offensive cybersecurity, is well within the technical and financial capabilities of many cyber criminals. There's just no way out of that.
Classic “rules for thee and not for me.”
Agreed that this could be reinjected later. Makes me think of neo in the matrix:
“I know zero days.”
“Show me.”
What "rules"? They don't have to post-train the model on anything specific, it's their judgment call.
The "rule" here is, if you read the disclosure carefully, that they have cyber trained models in house. But you do not get them, you only get the non-cyber models.
Post-training _is_ a part of the whole training regime.
Governments also control who owns tanks and missiles.
Do you think that going around making cyber weapons isn't going to get you buried in a shallow grave?