Comment by 7thaccount

2 years ago

I don't think this isn't what the reproducibility issue is about.

The problem is there is a broken system in place that rewards publishing in high volumes, only getting positive results, and also doing research that supports those in charge. That doesn't mean new ideas don't win out when clear evidence exists but it is the exception.

All of this leads to economic incentives to lie with the data and then you get to scientific papers that people are making decisions based off (like where to put research dollars for Alzheimers) that are fraudulent. This is not good at all.

4 comments

7thaccount

Reply
llamaimperative  2 years ago

There are multiple reproducibility crises. The incentive structure is one of them, the other being the general challenge of reproducing results further "up the stack" in highly chaotic fields (the stack being physics -> chem -> bio -> psychology -> sociology, obviously nonexhaustive)

I actually find the reproducibility crisis you're mentioning to be both more problematic and to receive far less attention than the other. More problematic because it infects every form of science, even the harder/more fundamental sciences, and because it's way less clear how to fix it.

The reproducibility crisis near the top of the stack is just: "do more science and don't believe results until they've been replicated, refined, and matured."

  • 7thaccount  2 years ago

    Ah gotcha. Good point.

    I've always felt that the psychology/sociology studies of the week I hear about on the radio were likely based on some weak statistics in the best of cases and most likely difficult to reproduce.