Comment by johncassil
17 hours ago
Good question -- I have been pointing claude at repos for security reviews before I install stuff now for exactly this reason.
17 hours ago
Good question -- I have been pointing claude at repos for security reviews before I install stuff now for exactly this reason.
It's so funny to read these, as if countless examples didn't prove that llms don't actually distinguish between your instructions and the data they read at your request.
Do you think a malicious repo author won't think of hiding a suitable prompt within?
> countless examples
Show me just one where a frontier model got prompt injected when checking if code if malicious.
FWIW I can do this, but I bet you can't.
Here's prompt injection via a harness for example: https://www.promptarmor.com/resources/google-antigravity-exf...
took me about 30 seconds to find. there's tons of them. of course it's an arms race, the models get better the harnesses get better the attackers get better but again: for the model itself, your prompt and third party data it reads are the same thing (input). as long as it is true, the attack will always be possible.
1 reply →