Comment by SilverBirch
1 year ago
Absolutely, I remember talking about this a while ago about one of the other image generation tools. I think the prompt was like "Generate an American person" and it only came back with a very specific type of American person. But it's like... what is the right answer? Do you need to consult the census? Do we need the AI image generator to generate the exact demographics of the last census? Even if it did, I bet you it'd generate 10 WASP men in a row at some point and whoever was prompting it would post on twitter.
It seems obvious to me that this is just not a problem that is solvable and the AI companies are going to have to find a way to justify the public why they're not going to play this game, otherwise they are going to tie themselves up in knots.
But there are thousands of such ambiguities that the model resolves on the fly, and we don't find an issue with them. Ask it to "generate a dog in a car", and it might show you a labrador in a sedan in one generation, a poodle in a coupe in the next, etc. If we care about such details, then the prompt should be more specific.
But, of course, since race is a sensitive topic, we think that this specific detail is impossible for it to answer correctly. "Correct" in this context is whatever makes sense based on the data it was trained on. When faced with an ambiguous prompt, it should cycle through the most accurate answers, but it shouldn't hallucinate data that doesn't exist.
The only issue here is that it clearly generates wrong results from a historical standpoint, i.e. it's a hallucination. A prompt might also ask it to generate incoherent results anyway, but that shouldn't be the default result.
But this is a misunderstanding of what the AI does. When you say "Generate me diverse senators from the 1800s" it doesn't go to wikipedia, find out the names of US Senators from the 1800s, look up some pictures of those people and generate new images based on those images. So even if it generated 100% white senators it still wouldn't be generating historically accurate images. It simply is not a tool that can do what you're asking for.
I'm not arguing from a technical perspective, but from a logical one as a user of these tools.
If I ask it to generate an image of a "person", surely it understands what I mean based on its training data. So the output should fit the description of "person", but it should be free to choose every other detail _also_ based on its training data. So it should make a decision about the person's sex, skin color, hair color, eye color, etc., just as it should decide about the background, and anything else in the image. That is, when faced with ambiguity, it should make a _plausible_ decision.
But it _definitely_ shouldn't show me a person with purple skin color and no eyes, because that's not based in reality[1], unless I specifically ask it to.
If the technology can't give us these assurances, then it's clearly an issue that should be resolved. I'm not an AI engineer, so it's out of my wheelhouse to say how.
[1]: Or, at the very least, there have been very few people that match that description, so there should be a very small chance for it to produce such output.