Comment by stickfigure
4 days ago
Unanswered by the article: Does any of that (joke or not) actually work?
It's input to the model, sure, but it would have to be a pretty dumb model.
I don't doubt that it's possible to create prompt injection attacks that work. I'm very interested in knowing how hard it is.
My intuition is that the models from this year are really hard to successfully inject.
The labs have been pouring a ton of energy into training against these attacks, as hinted from their system cards.
Doesn't mean they can't be beaten, but you likely need a whole lot more context than a short HTTP header to do it.
It would be very surprising if they weren't working on even more long-context injection, an agent these days could very easily have a single http call get the markdown/http for a whole page that's essentially injection/propaganda. It seems like the agents have been trained on wrapping responses in tails/cats from `.log` calls or other tool calls, in which case the model training will implicitly be trained on the trustworthiness of context within the tool call tokens.
Testing now.
[flagged]