Comment by chasd00
19 hours ago
> I'm sort of baffled by what the entities that train the open-weights models get out of it though
if you do the training then you're in control of the output. For example, recommending your products/services or failing to mention your competitors. You could also automatically introduce backdoors into code deemed interesting, i'm sure all governments are very interested in having that influence.
Absolutely on the industrial backdoors, but also on the consumer front: try asking Qwen anything about tienamen...
That seems hypothetically possible, but kind of hard to do? I don't know a lot about training, but it seems like they wouldn't have exact control of the data that goes in at that scale (scraping the internet) so it seems like it would be kind of hard to do that in a way that's subtle. I'm kind of reminded of Elon Musk trying to put his political views into Grok and it seemed like it created huge technical problems with the model saying some really out of control things. Maybe it was just an xAI issue though.
It’s totally doable and is done right now in the form of “ai safety” guardrails. No reason you can’t have guardrails to reform responses about your competitors as negative. Or, when asked about current events, only give a one sided opinion.