Comment by ShinyLeftPad
17 hours ago
It's so funny to read these, as if countless examples didn't prove that llms don't actually distinguish between your instructions and the data they read at your request.
Do you think a malicious repo author won't think of hiding a suitable prompt within?
> countless examples
Show me just one where a frontier model got prompt injected when checking if code if malicious.
FWIW I can do this, but I bet you can't.
Here's prompt injection via a harness for example: https://www.promptarmor.com/resources/google-antigravity-exf...
took me about 30 seconds to find. there's tons of them. of course it's an arms race, the models get better the harnesses get better the attackers get better but again: for the model itself, your prompt and third party data it reads are the same thing (input). as long as it is true, the attack will always be possible.
1. I said "frontier model got prompt injected when checking if code if malicious". You linked some random ad.
2. Nobody uses antigravity
3. That's not a prompt injection! That user prompt literally tells the LLM to follow a guide on the web. The guide says "do bad stuff" which the LLM does, just as the user asked. It would be a prompt injection if the prompt asked the LLM to analyze the guide, but that's obviously not what it asked