Comment by xyzsparetimexyz
7 days ago
Is this possible to detect in any kind of smart way? Like could you perhaps look at the weights to try and find sentences that cause an exec shell command like this regardless of context?
7 days ago
Is this possible to detect in any kind of smart way? Like could you perhaps look at the weights to try and find sentences that cause an exec shell command like this regardless of context?
There was a link in another thread on HN yesterday to this:
https://www.crowdstrike.com/en-us/blog/crowdstrike-researche...
That's about Deepseek, I would expect every model out there, closed or open to have tricks on board like that. If you run a model without safeguards in place you have essentially RCE'd yourself.