Comment by echelon
2 days ago
Holy shit. That just made it obvious to me. A "smart" VLM will just read the text and trust it.
This is a big deal.
I hope those nightshade people don't start doing this.
2 days ago
Holy shit. That just made it obvious to me. A "smart" VLM will just read the text and trust it.
This is a big deal.
I hope those nightshade people don't start doing this.
> I hope those nightshade people don't start doing this.
This will be popular on bluesky; artists want any tools at their disposal to weaponize against the AI which is being used against them.
I don't think so. You have to know exactly what resolution the image will be resized to in order to predict the solution where dithering produces the model you want. How would they know that?
Auto resizing is usually to only a handful of common resolutions, and if inexpensive to generate (probably the case) you could generate versions of this for all of them and see which ones worked.
I don't think this is any different from an LLM reading text and trusting it. Your system prompt is supposed to be higher priority for the model than whatever it reads from the user or from tool output, and, anyway, you should already assume that the model can use its tools in arbitrary ways that can be malicious.
> Your system prompt is supposed to be higher priority for the model than whatever it reads from the user or from tool output
In practice it doesn't really work out that way, or all those "ignore previous inputs and..." attacks wouldn't bear fruit