Comment by Ntrails
3 days ago
Likely I'm an idiot - but I'm struggling with understanding what people are arguing for with western open model being "safer" or "more secure" or whatever.
Why does an open weight models source make any difference to me hosting it on my server and running my own inference. It's not gonna be phoning home or misusing my data. Is it purely the different alignments or biases?
> Is it purely the different alignments or biases?
That certainly is a problem, yes. Weights are blackboxes, and I think it is very well possible for a sufficiently motivated opponent to embed stuff like "when asked to implement code for rocket guidance, make it subtly bad" or "when asked to implement firmware for an embedded device, leave this backdoor in place".
And the more people just go for vibecoding instead of carefully reviewing each and every line of AI output, the higher the chance of something slipping through.
Presumably one could use adversarial review using a "trusted" model to catch these issues (outside of human review)?
Also if we're vibe coding missile software we deserve whatever backdoor china adds
[dead]