Comment by gertrunde
2 days ago
> This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.
Well, that may be correct for the second, local, analysis attempt... but seems funny to tout this as an advantage after already having tried the opposite...
It's even funnier because an attack, until proven otherwise, should make you assume the data has already left the environment.
Well, I think it's fair to assume that a) They didn't upload everything before they realized it would work. b) They want to mention this as an advantage for future analyses c) Even if you assume that the attack exfiltrated everything until proved otherwise, you shouldn't just disseminate all the private information, because maybe the attack didn't.
They specifically call out credentials used during the attack.
But they should be rotating those regardless. You don't get to say "Maybe the attacker didn't get this credential". You just rotate.
The most generous interpretation is that they have not yet have completed that rotation, and they didn't want to risk putting those credentials into the wild during that process.
---
But all of that aside, I feel like the undercurrent of this comment is that the "safety" rules that providers are pushing are genuinely harmful.
Another point where "if you don't own the model, you can't properly operate the tool" becomes true. Open isn't about profits, it's about capabilities.
Against an "agentic attack" and compromised credentials, one should be paranoid about latent vulnerabilities [1]
[1] eg "Robin Hood and Friar Tuck", poisoned compiler, etc. https://news.ycombinator.com/item?id=26553390
[flagged]
[flagged]