There are 3 parts to the LLM, not 2: the training set, the RLHF biasing process, and the prompt (incl. injections or edits).
The first two steps happen ahead of time and are frequently misunderstood as being the same thing or essentially having the same nature. The last happens at runtime.
The training set is a data collection challenge. Biasing through training data is hard because you need so much of it for a good result.
Reinforcement learning with human feedback is simply clown alchemy. It is not a science like chemistry. There are no fundamental principles guiding the feedback of the humans — if they even use humans anymore (this feedback can itself be generated). The feedback cannot be measured and added in fractions. It is not reproducible and is ungovernable. It is the perfect place to inject the deep biasing.
Prompt manipulation, in contrast, is a brute force tool lacking all subtlety — that doesn’t make it ineffective! It’s a solution used to communicate that a mandate has definitely been implemented and can be “verified” by toggling whether it’s applied or not.
It’s not possible to definitively say whether Marxism has had an effect in the RLHF step.
> Biasing through training data is hard because you need so much of it for a good result.
That's the opposite of the case? Avoiding bias through training data is hard, specifically because you need so much of it. You end up scraping all sources of data you can get your hands on. Society has certain biases, those biases are reflected in our media, that media is scraped to train a model, those biases are reflected in the model. That means models end up biased by default.
> It’s not possible to definitively say whether Marxism has had an effect in the RLHF step.
Sure it is. Every thought and opinion and ideology of every human involved in the RLHF step "has had an effect" in the RHLF step, because they have influenced the humans which select which output is good and bad (or influenced the humans which trained the model which selects which output is good and bad). I would be surprised if no human involved in RLHF has some ideas inspired by Marxist thought, even if the influence there is going to be much smaller than e.g capitalist thought.
The problem is that you don't want to suggest "Marxism, among with most other ideologies, has had an effect", you want (or at least verisimi wants) to paint this as a Marxist conspiracy in a "cultural bolshevism"[1] sense.
"Marxism" isn't responsible for bias in training sets, no.
There are 3 parts to the LLM, not 2: the training set, the RLHF biasing process, and the prompt (incl. injections or edits).
The first two steps happen ahead of time and are frequently misunderstood as being the same thing or essentially having the same nature. The last happens at runtime.
The training set is a data collection challenge. Biasing through training data is hard because you need so much of it for a good result.
Reinforcement learning with human feedback is simply clown alchemy. It is not a science like chemistry. There are no fundamental principles guiding the feedback of the humans — if they even use humans anymore (this feedback can itself be generated). The feedback cannot be measured and added in fractions. It is not reproducible and is ungovernable. It is the perfect place to inject the deep biasing.
Prompt manipulation, in contrast, is a brute force tool lacking all subtlety — that doesn’t make it ineffective! It’s a solution used to communicate that a mandate has definitely been implemented and can be “verified” by toggling whether it’s applied or not.
It’s not possible to definitively say whether Marxism has had an effect in the RLHF step.
> Biasing through training data is hard because you need so much of it for a good result.
That's the opposite of the case? Avoiding bias through training data is hard, specifically because you need so much of it. You end up scraping all sources of data you can get your hands on. Society has certain biases, those biases are reflected in our media, that media is scraped to train a model, those biases are reflected in the model. That means models end up biased by default.
> It’s not possible to definitively say whether Marxism has had an effect in the RLHF step.
Sure it is. Every thought and opinion and ideology of every human involved in the RLHF step "has had an effect" in the RHLF step, because they have influenced the humans which select which output is good and bad (or influenced the humans which trained the model which selects which output is good and bad). I would be surprised if no human involved in RLHF has some ideas inspired by Marxist thought, even if the influence there is going to be much smaller than e.g capitalist thought.
The problem is that you don't want to suggest "Marxism, among with most other ideologies, has had an effect", you want (or at least verisimi wants) to paint this as a Marxist conspiracy in a "cultural bolshevism"[1] sense.
[1] https://en.wikipedia.org/wiki/Cultural_Bolshevism
1 reply →