Comment by d0mine
2 years ago
It is worth pointing out that compared to * everything else on the internet, science is the epitome of truth, facts, reproducibility, etc.
It might not appear so at the first glance due to Gell-Mann Amnesia effect.
2 years ago
It is worth pointing out that compared to * everything else on the internet, science is the epitome of truth, facts, reproducibility, etc.
It might not appear so at the first glance due to Gell-Mann Amnesia effect.
I don't think this isn't what the reproducibility issue is about.
The problem is there is a broken system in place that rewards publishing in high volumes, only getting positive results, and also doing research that supports those in charge. That doesn't mean new ideas don't win out when clear evidence exists but it is the exception.
All of this leads to economic incentives to lie with the data and then you get to scientific papers that people are making decisions based off (like where to put research dollars for Alzheimers) that are fraudulent. This is not good at all.
There are multiple reproducibility crises. The incentive structure is one of them, the other being the general challenge of reproducing results further "up the stack" in highly chaotic fields (the stack being physics -> chem -> bio -> psychology -> sociology, obviously nonexhaustive)
I actually find the reproducibility crisis you're mentioning to be both more problematic and to receive far less attention than the other. More problematic because it infects every form of science, even the harder/more fundamental sciences, and because it's way less clear how to fix it.
The reproducibility crisis near the top of the stack is just: "do more science and don't believe results until they've been replicated, refined, and matured."
Ah gotcha. Good point.
I've always felt that the psychology/sociology studies of the week I hear about on the radio were likely based on some weak statistics in the best of cases and most likely difficult to reproduce.
Fittingly for this stack model, maths rarely (but not never!) has reproducibility issues.
https://xkcd.com/435/
1 reply →
The practice of the scientific method is the path to empirical truth. This doesn't necessarily mean that a nominal scientific finding is true. It is likely true that, on a given topic, an acredited scientist in that field has fairly correct opinions, but I would not take this too far. Conflicts of interest, personal biases, incentives to obtain results, simple lack of reproduction, corrupt peer review, etc. are clearly issues, and it's all the more unfortunate that we can't even say just how deep these issues run.
Science is timeless and powerful. Scientists are human beings that nominally, preferably, fallibly practice science.
Gell-Mann amnesia is when you notice that somebody who is writing about something you're an expert in has made critical mistakes, and should not be trusted, but then you go on and trust people writing about topics you don't know as much about, failing to recognize they're probably also making critical mistakes about those other topics. I'm not sure the connection here.
Regarding the comparison between science and folk explanations on the internet, I think it's reasonable to hold science to a higher standard than we hold the general population. If most published research findings are false—as this video claims—and most of what idiots on the internet say is also false, that's neither an equivalence nor a victory for science. On the contrary, it erodes the value of science, both literally and figuratively. Right now we need sources of authority.
It is not black or white. Yes, there are issues. No, it doesn’t mean a random comment on the internet should have as much authority as scientific paper. There is a strong background of anti-intellectualism right now.
The connection with Gell-Mann amnesia is that people overestimate truthiness of what they read online. Combined with reading about [real] issues in science, It might create an impression that scientific findings are relatively at the same level as everything else. My point is even all the troubles science is still a head above despite the perception.