Comment by Aurornis
7 days ago
When the models are used in harnesses that allow tool calls people default to approving actions by default, or clicking through the approvals.
Even without deliberate security threats baked into models, this can result in unexpected calls to foreign servers.
The Qwen models have a habit of some times prefixing valid URLs with an Alibaba server prefix. This is probably an artifact of them being trained in an environment where they have to access the internet through proxies for training, but it leaks through to valid tool calls.
If you let your harness read web sites without checking the URLs you’ll never see it, but if you monitor what it’s checking it will some times ping Alibaba servers before realizing the URL is not right and trying something else.
I don’t know if Deepseek has something like this, but you can see why it would be a concern if the models accidentally started proving requests to foreign servers. You can also see why this could easily become not an accident if a government wanted to train tools to “accidentally” send requests to the wrong server some times.
No comments yet
Contribute on Hacker News ↗