Comment by brucethemoose2
3 years ago
What the article doesn't illustrate is that it destroys fine detail in the image, even in the thumbnails of the reference paper: https://arxiv.org/pdf/2310.13828.pdf
Also... Maybe I am naive, but it seems rather trivial to work around with a quick prefilter? I don't know if tradition denoising would be enough, but worst case you could run img2img diffusion.
reply
The poisoned images aren't intended to be viewed, rather scraped and pass a basic human screen. You wouldn't be able to denoise as you'd have to denoise the entire dataset, the entire point is that these are virtually undetectable from typical training set examples, but they can push prompt frequencies around at will with a small number of poisoned examples.
> You wouldn't be able to denoise as you'd have to denoise the entire dataset
Doing that requires much less compute than training a large generative image model.
I guess the idea is that the model trainers are ignorant of this and wouldn't know to preprocess/wouldn't bother?
That's actually quite plausible.
2 replies →
> the entire point is that these are virtually undetectable from typical training set examples
I'll repeat this point for clarity. After going over the paper again, denoising shouldn't affect this attack, it's the ability of plausible images to not be detected by human or AI discriminators (yet)