Comment by kelipso
2 days ago
There very likely is existing research into evaluating political bias in LLMs, not too sure, but I do think it's very possible to have an evaluation framework that could test LLMs for political bias and other biases. Once we have such a test and an LLM that passes it, we can be certain (to some confidence, for some topics, for some biases, etc etc) that the LLM won't be biased.
For humans, there is no such guarantee. The humans can lie, change their mind, etc. See Wikipedia, where they talk about how they are not biased, they have many processes that ensure no biases, blah blah blah, and it turns out they are massively biased, what a surprise.
Of course, who evaluates the evaluators/evaluation frameworks comes into play but that's a much easier problem.
> See Wikipedia, where they talk about how they are not biased, they have many processes that ensure no biases, blah blah blah, and it turns out they are massively biased, what a surprise.
It's clear you have some unfounded issue with Wikipedia. They are not "massively biased", that's a talking point propelled primarily by the right/far right because of a desire to rewrite history to match their ideological needs.
Saying "there very likely is existing research into evaluating political bias in LLMs" essentially means very little because
1. By your own admission you can't even say for sure that such research is actually happening (it probably is, but you admit you don't actually know) 2. There is no guarantee such research will lead to anywhere anytime soon 3. Even if it does, how does a means of evaluating bias in LLMs provide a path to eliminating it?
It’s not “unfounded”. Wikipedia is biased and saying that’s “propaganda” or a result of propaganda is a nonsense non-argument.
> Saying "there very likely […]
What’s with this nitpicky stuff. A simple google search shows there’s tons of research in LLM political bias evaluation.
> There is no guarantee [..] path to eliminating it?
It’s research. Sure there’s no guarantee but given progress in LLM, I would be optimistic rather than pessimistic.
> It’s not “unfounded”. Wikipedia is biased and saying that’s “propaganda” or a result of propaganda is a nonsense non-argument.
It specifically is unfounded if you have no credible sources to back it up. "Trust me bro" doesn't qualify.
> What’s with this nitpicky stuff
This is HN, you should be prepared to validate what you're saying, or accept you'll be challenged to do so.
> It’s research. Sure there’s no guarantee but given progress in LLM, I would be optimistic rather than pessimistic.
This is a really poor argument when advocating it (AI) as a viable replacement for the status quo.
2 replies →