Comment by Loughla
5 years ago
That is my number one pissing point right now in higher education.
Every company has a predictive algorithm to use on students. Every startup that's stepping into the space is pushing the data and data scientists.
But they all have the same-old, usually decades old, baked in biases. AND they're not doing anything to address it!
Just because it's math doesn't mean it's not biased. I hate it more than anything professionally, right now.
> Just because it's math doesn't mean it's not biased.
When a model produces an unpalatable result, that doesn't mean it is biased. All these algorithmic fairness people are saying, once you peel back the layers of rhetorical obfuscation, is that we should make ML models lie. Lying helps nobody in the long run.
>When a model produces an unpalatable result, that doesn't mean it is biased.
Absolutely, but there is no admission, from what I can tell, from ML or predictive [buzzword here] companies that bias is a thing, or even could be a thing in their systems.
>All these algorithmic fairness people are saying, once you peel back the layers of rhetorical obfuscation, is that we should make ML models lie. Lying helps nobody in the long run.
Maybe I'm misunderstanding, but that is not at all what I am saying as an 'algorithmic fairness' person. I am saying that we need to ensure there are strict oversights and controls on the building/execution of algorithms when making substantive decisions about human people.
For example: It's okay if an algorithm predicting student success says that all the minority students on my campus are at a higher risk of dropping out. That is a data point. Historically, minority students drop out at a higher rate. Sure. Not great, but it is factually true.
What is not okay is for the 'predictive analytics' company to sell their product in conjunction with a 'tracking' product that limits minority students' access to selective admissions programs simply because they are selective, more difficult, and, historically, have a higher percent of minority students who drop out.
I guess what I'm saying is that ML models shouldn't lie. But they also shouldn't be seen as the truth above all truths. Because they're not. They're just data, interpreted through the lens of whoever built the models.
Every human carries a bias, everyone. It's how we define ourselves as 'self' and others as 'other' at a basic level.
Therefore, everything we build, especially when it's meant to be intuitive, may carry those biases forward.
I'm only saying we need to be aware of that, acknowledge it, and ensure there are appropriate controls and oversight to ensure the biases aren't exasperated inappropriately.
What do you think about the second issue : the "blackboxing" ?
Oh wow, this is so juvenile.
The problem with these ML models is that they are directly connected to a response, and because they use statistics and math they are simplistically perceived as truth. They're not truth, they're are nothing more than models. The truth can fit a plethora of models. Ignoring human bias while training ML models, and then just saying that the model is truth is exactly the problem.
Thank you for demonstrating the issue so vividly.
> Oh wow, this is so juvenile.
Please try to elevate the debate. See [1]. You're at DH0 right now.
> They're not truth, they're are nothing more than models. The truth can fit a plethora of models.
Models receive past data and emit predictions. We can then see how well those predictions match future data. We call one model "better" than another model when that first models' predictions more closely match future data than the second model's. Not all models are equivalent. The ML fairness people want to make model predictions less accurate because they don't like what the predictions say. Prioritizing truth over pleasantness isn't juvenile: it's the opposite. The mark of maturity is the willingness to accept an unpleasant reality instead of denying it.
[1] http://www.paulgraham.com/disagree.html