Comment by jfaat
7 days ago
Certainly? I don’t have time to read the whole thing but-
> we found that when DeepSeek-R1 receives prompts containing topics the CCP likely considers politically sensitive, the likelihood of it producing code with severe security vulnerabilities increases by up to 50%.
So a relatively ancient model got tripped up in latent space gray area it produced sloppier code and crowdstrike is all like “gotcha!” Sounds like the basic thesis? Not great support, more like FUD
Note that all of a sudden, the great majority of the comments on this page appear to be pro-US government and pretend that the CCP is somehow worse. Not a coincidence: the regulatory capture is starting right here.
It's a banal observation that users of a US-based venture capital firm's web forum have a bias toward that country's governance model.
Honored at the suggestion that I'm a pro-US shill, thanks. :)
You're certainly acting like one, by responding to the parent comment in that manner and then to me after I wrote that. I have no idea who you are.
Such gray areas are hallmarks of alignment post-training (aka intentional sabotage).
"No fair, you haven't caught me lately!"
I mean you keep using the words intentional and sabotage but you provide 0 proof of either. This isn't a political thing to me- I'm not defending Xi here. I'm defending logic. Show me a single example of this sabotage in the wild, and I'll gladly consider changing my mind. Show me any proof of intention and I'll do the same. That's supposed to be the bar here, isn't it? That it's an old and extremely weak model by today's standards is material. I wasn't crying foul because it's not aligned with the reality I want.
"Sabotage" is trivial. You mention XYZ, performance degrades. This is (by definition) sabotage against XYZ, whether intentional or otherwise.
"Intentional" is an interesting point to press me on. No respondant has suggested that China didn't intentionally "align" Chinese models against the US (and Tibet, and Uyghur Muslims, and...). The only objectioN has been "but you also ~~lynch black people~~ align your LLMs against us." Classic Soviet-style whataboutism.
Which reality is that? Please be very specific. :)
1 reply →
If they can reproduce scenarios where code is of lower quality when certain topics or contexts are injected, then that is a strong signal of intentional design, isn't it?
No; it’s a signal of lobotomy. Models get dumber near where you sensor them. You could almost think of it like your prompt is getting redacted, and so the LLM has to deal with these holes in your prompt (where you mentioned the semantic contraband) and it doesn’t do a very good job because it has no idea what you’re talking about.
The mechanism is likely a bit inverted, it’s the weights that have been redacted, but the effect is that weights that were interfered with are affecting the problem solving abilities of the LLM - I mean, it’s insecure bad code, not a backdoor, right?
Labeling it "censorship" or "alignment training" doesn't change what it is: intentional anti-US sabotage.
"Oops, I meant to train my model to be aligned against the US, but woah, now it accidentally harms the US!! My bad!!"
Not convincing, sorry.
Six of one and half-dozen of the other.
Since the agent can't necessarily phone home, the best it can do is scatter vulnerabilities everywhere (to be found later by LLMs in China).
If you prefer, think of it as closer to stochastic industrial sabotage[0] rather than pre-planning a particular backdoor in particular software. Different tools, different tactics!
[0] https://news.ycombinator.com/item?id=35448090
I would have to clarify the test mechanisms of crowdstrike. I assumed it was prompting for info on censored topics, THEN asking for code development.
What you're suggesting, to use a metaphor, is that it's like the LLM hearing a censored topics and getting the jitters, rather than it hearing a censored topics and deciding to act maliciously.