Comment by mort96
1 year ago
> Biasing through training data is hard because you need so much of it for a good result.
That's the opposite of the case? Avoiding bias through training data is hard, specifically because you need so much of it. You end up scraping all sources of data you can get your hands on. Society has certain biases, those biases are reflected in our media, that media is scraped to train a model, those biases are reflected in the model. That means models end up biased by default.
> It’s not possible to definitively say whether Marxism has had an effect in the RLHF step.
Sure it is. Every thought and opinion and ideology of every human involved in the RLHF step "has had an effect" in the RHLF step, because they have influenced the humans which select which output is good and bad (or influenced the humans which trained the model which selects which output is good and bad). I would be surprised if no human involved in RLHF has some ideas inspired by Marxist thought, even if the influence there is going to be much smaller than e.g capitalist thought.
The problem is that you don't want to suggest "Marxism, among with most other ideologies, has had an effect", you want (or at least verisimi wants) to paint this as a Marxist conspiracy in a "cultural bolshevism"[1] sense.
The ambient bias in the training data is not a concern. The directional bias that can be inflicted during the RLHF step consumes most of my concern.
How? Simply by putting the right types of people onto the task! Don’t you know that the human participants in RLHF processes are screened? Will the feedback provided by homogeneous collections of Ultra MAGA Trumpers, Woke Zealots, or WEF Sycophants result in an unbiased model? The same model?
Do we know who provided feedback to Gemini? Do we know what they were told, promised, or paid?
Only Google HR knows.