Comment by sega_sai
1 year ago
That is certainly embarassing. But in the same time, I think it is a debate worth having. What corrections to the training dataset biases are acceptable. Is it acceptable to correct the answer to the query "Eminent scientist" from 95% men, 5% woment to 50%/50% or to the current ratio of men/women in science ? Should we correct the ratio of black to white people in answering a generic question to average across the globe or US ?
In my opinion, some corrections are worthwhile. In this case they clearly overdone it or it was a broken implementation. For sure there will be always people who are not satisfied. But I also think that the AI services should be more open about exact guidelines they impose, so we can debate those.
> Is it acceptable to correct the answer to the query "Eminent scientist" from 95% men, 5% woment to 50%/50% or to the current ratio of men/women in science ? Should we correct the ratio of black to white people in answering a generic question to average across the globe or US ?
I would expect AI to at least generate answers consistent with reality. If I ask for a historical figure who just happens to be white, AI needs to return a picture of that white person. Any other race is simply wrong. If I ask a question about racial based statistics which have an objective answer, AI needs to return that objective answer.
If we can't even trust AI to give us factual answers to simple objective facts, then there's definitely no reason to trust whatever AI says about complicated, subjective topics.
I agree. For specific historical figures it should be consistent with reality. But for questions about broad categories, I am personally fine with some adjustments.
> I would expect AI to at least generate answers consistent with reality
Existing services hallucinate all the time. They can't even do math reliably, nor can you be reasonably certain it can provide actual citations for any generated facts.
Yep, and I would say more broadly speaking if I ask for pictures of vikings, I would expect 100% of them to be white.
We aren't talking about ratios here. The ratio is 100% not white, no matter what you ask for. We know it's messed up bad because it will sometimes verbally refuse to generate white people, but it replies enthusiastically for any other race.
If people are getting upset about the proportion of whatever race in the results of a query, a simple way to fix it is to ask them to specify the number and proportions they want. How could they possibly be offended then? This may lead to some repulsive output, but I don't think there's any point trying to censor people outside of preventing illegal pornography.
I think it is clear that it is broken now.
But thinking what we want is worth discussing. Maybe they should have some diversity/etnicity dial with the default settings somewhere in the middle between no correction and overcorrection now.
[dead]
It is 100% white if you ask for something that is a negative stereotype, like "a family eating fried chicken".
Why is bias a problem?
When you prompt "business man" and it outputs a white man, this is quite probably reflective of representation in reality.
Whether this overrepresentation is even a problem at all is debatable as the idea that every job, role or subgroup of people is perfectly diverse or that this even should be the goal isn't just ridiculous, it's demographically impossible.
If you do have a problem with a specific representation in actual reality, reality itself should change. Which it does, it just takes time.
In the meanwhile, just prompt "black business man" if that's what you were after.
Which reality? For sure "business man" being 100% white is not a reality for some people in Africa and Asia, don't you think?
Google barely makes any money from Africa (relatively speaking). They are a non-factor from a shareholder standpoint. From a shareholder standpoint, the North American, European, Australian, and Japanese markets are top of mind.
1 reply →
>Is it acceptable to correct the answer to the query "Eminent scientist" from 95% men, 5% woment to 50%/50% or to the current ratio of men/women in science ? Should we correct the ratio of black to white people in answering a generic question to average across the globe or US ?
It’s a great question, and one where you won’t find consensus. I believe we should aim to avoid arrogance. Rather than prescribing a world view, prescribe a default and let the users overwrite. Diversity vs. reality should be a setting, in the users’ control.
I think baseline it should be tuned to be statistically accurate. The problem is that people leave a lot in their prompt to be implied. Some interaction designers use this as an opportunity to infill their own subjective opinion of what should be inferred as a way to 'take care' of the user.
This isn't their only option... they could also just ask for more information.
A good approach here would be ask the user to further clarify what exactly they want before generating a person — "Do you want a random depiction or a specific depiction". A good tool for users is one which helps them be and feel more tactically or predictably in control of it; which means making them aware of its behavioural pitfalls so they can avoid them if they want to.
> In my opinion, some corrections are worthwhile.
The problem with “corrections” is that they obscure the truth. If you’re being given information and start forming perceptions that no longer map onto reality you’re actually in a much worse position to change or do anything about reality itself. It’s basically like you’re being lied to and misled. How can you fix the situation if you don’t even have the facts at hand or you’re not being made aware of the facts?
This problem has long been solved. Who decides what's correct? No one. Once you start censoring (censor, from Latin censere meaning ‘assess’) you're already sliding down the slope.
Yet humans are doomed to forget and relive history.
Good question. What do you “correct” and what not? Where do you draw the line? Isn’t any line arbitrary?
It seems truth is the only line that isn’t arbitrary.
There is AI bias. I think the most common scenario on Dall-e prior to the "fixes", was to ask for "doctors" and only get white people. Never black.
The thing is, you don't fix this by changing the user prompt. You fix this by "removing" the bias on your dataset!
Removing under quotes because of course you are just changing to another accepted bias.
"Without regard for race" seems sound in law. Why should those writing the code impose any of their racial views at all? When asked to generate an image of a ball, is anyone concerned about what country the ball was made in? If the ball comes out an unexpected color, do we not just modify our prompts?