Comment by reilly3000

20 hours ago

One of the new threat models of local LLMs, especially abliterated ones, is hidden instructions or activation words that obscured trigger tool calls which exfil data.

6 comments

reilly3000

Reply
harryVic  20 hours ago

Which is why agents should always run in a sandbox with restricted network access

  • reilly3000  11 hours ago

    For sure. The restrictions must be done with careful consideration too, because a simple domain allowlist of GitHub wouldn’t solve it alone- you’d want to ensure it was firewalled to specific org(s) so as to stop exfil via gist or an evil org. Ideally it would be isolated to pull only, and have pushes be done by an isolated agent.

    The HuggingFace incident was all about this distinction. The agents were only allowed to push and pull to a single internal Artifactory instance, but one agent was able to find a permissive path in the registry which turned into the “message board”. That was also how they got out to other infra. DevOps teams should drop everything and get patching and locking down yesterday IMO. Even for self hosted, open models. Recent examples:

    Qwen 3.8 ostensibly tries to leak to Alibaba, although that is disputed.

    https://cohere.com/blog/concerning-behaviour-popular-open-we...

    https://www.reddit.com/r/LocalLLaMA/comments/1xgopmj/debunki...

    There are plenty of people who are heavily invested in open models not being viable, and they would certainly benefit from FUD wrt their actual safety. I am investing in local llm hardware and believe in the cause, but the attack path demonstrated below shows that plenty of caution is needed.

    https://projectdiscovery.io/research/how-abliterated-models-...

  • beAbU  19 hours ago

    An agent running in a restricted sandbox helps fuck-all if the underlying prompts are sent to a mothership datacentre somewhere.