Comment by lelanthran

> It seems much more interesting to look for biases that appear robustly across different realistic scenarios that would actually be influenced by the training data

Difficult to do when you're following a scientific process: you want to keep all confounding variables the same while varying only the single one that you are measuring.

Measuring realistic scenarios (say, using real race names, or real cities, etc) doesn't give a decent result because any bias you see might be bias in the training data.

TBH, they shouldn't have used real roles/positions like "doctor", either.