The problem with nightshade is that while it worked in laboratory conditions, in practical application it wouldn't work without extensive coordination between everyone using it.
For example, if one artist who draws dogs uses it to bias towards cats and another uses it to bias towards horses, the bias data is less of a signal and more noise. In the paper, they biased all the images the same way.
The issue compounds when you consider the multiple data points that needs to be biased. An artist who draws impressionist dogs, an artist who draws cartoon dogs, and an artist who draws cartoon cats would need to bias 'impressionism,' 'cartoon style,' 'dogs,' and 'cats' all in ways that are standardized across nightshade users to have the tool be effective.
This isn't realistically going to happen.
So ultimately it's about as effective as the users who put the "you don't have permission to use my data" clauses on their MySpace two decades ago. Feels good, and totally ineffective.
Interesting, but that doesn't really answer my question. Has anyone done wider studies about the effectiveness of this sort of thing generally? As in, does it have to be tailored to a specific model or does it offer generalized protection with all models?
The Ars article doesn't talk about this aspect.
If these methods are effective, can a similar thing be done with text?
I can't speak to broader study of this, but I suspect this is only doable with images because you can make meaningful changes to the pixel data that remain mostly imperceptible to the eye. I think there's less room for this sort of thing in text.
"Feel good."
The problem with nightshade is that while it worked in laboratory conditions, in practical application it wouldn't work without extensive coordination between everyone using it.
For example, if one artist who draws dogs uses it to bias towards cats and another uses it to bias towards horses, the bias data is less of a signal and more noise. In the paper, they biased all the images the same way.
The issue compounds when you consider the multiple data points that needs to be biased. An artist who draws impressionist dogs, an artist who draws cartoon dogs, and an artist who draws cartoon cats would need to bias 'impressionism,' 'cartoon style,' 'dogs,' and 'cats' all in ways that are standardized across nightshade users to have the tool be effective.
This isn't realistically going to happen.
So ultimately it's about as effective as the users who put the "you don't have permission to use my data" clauses on their MySpace two decades ago. Feels good, and totally ineffective.
This article has some examples of output from poisoned vs clean models.
https://arstechnica.com/information-technology/2023/10/unive...
Interesting, but that doesn't really answer my question. Has anyone done wider studies about the effectiveness of this sort of thing generally? As in, does it have to be tailored to a specific model or does it offer generalized protection with all models?
The Ars article doesn't talk about this aspect.
If these methods are effective, can a similar thing be done with text?
I can't speak to broader study of this, but I suspect this is only doable with images because you can make meaningful changes to the pixel data that remain mostly imperceptible to the eye. I think there's less room for this sort of thing in text.
Happy to be proven wrong, though!
It depends. But poisoned images fed into a vulnerable training setup will scramble the output and make the model pretty useless.