Comment by laylower
20 hours ago
They never expected their models could be used for reverse engineering.That's the truth.
Now, they're panicking. And this is why the other dude, OpenAI, came out to say that people shouldn't use RAM locally, because they can't control the software anymore.
Open weight models, ablated models are going to become the norm, hopefully, locally run.
So instead, they're saying, look, hardware should become hard to reach for all of us, for the consumers.It's not because we're competing for resources for the same silicon or chips as they are.The ship has sailed thankfully.
One of the new threat models of local LLMs, especially abliterated ones, is hidden instructions or activation words that obscured trigger tool calls which exfil data.
Which is why agents should always run in a sandbox with restricted network access
For sure. The restrictions must be done with careful consideration too, because a simple domain allowlist of GitHub wouldn’t solve it alone- you’d want to ensure it was firewalled to specific org(s) so as to stop exfil via gist or an evil org. Ideally it would be isolated to pull only, and have pushes be done by an isolated agent.
The HuggingFace incident was all about this distinction. The agents were only allowed to push and pull to a single internal Artifactory instance, but one agent was able to find a permissive path in the registry which turned into the “message board”. That was also how they got out to other infra. DevOps teams should drop everything and get patching and locking down yesterday IMO. Even for self hosted, open models. Recent examples:
Qwen 3.8 ostensibly tries to leak to Alibaba, although that is disputed.
https://cohere.com/blog/concerning-behaviour-popular-open-we...
https://www.reddit.com/r/LocalLLaMA/comments/1xgopmj/debunki...
There are plenty of people who are heavily invested in open models not being viable, and they would certainly benefit from FUD wrt their actual safety. I am investing in local llm hardware and believe in the cause, but the attack path demonstrated below shows that plenty of caution is needed.
https://projectdiscovery.io/research/how-abliterated-models-...
An agent running in a restricted sandbox helps fuck-all if the underlying prompts are sent to a mothership datacentre somewhere.
3 replies →
For people touting about how their software can destroy the world, they aren't awfully creative on how it can achieve that. Hacking breaches of IP was probably most cynicists' top 3 questions asked when presented with the idea of "boosting productivity". If you make it "10x" faster to write code, you make it 10x faster for a hacker to hack. Naive cryptographic principles are less about an ironclad security and more about encryption that takes too much time to brute force. It all leads to one thing.
I'm still fairly adamant that those claims are mostly a myth. Because the worst inefficiencies in any workplace has little to do with the individual productivity of the proletariat. And because much of the most valuable work lies in creativity, ingenuity, and subtleties that lies outside of an LLM's expertise.
But loosely speaking, a lot of hacking does in fact come down to a lot of boring, repetitive tasks. Something a machine has always been king of.