Comment by Pooge
6 hours ago
Oh, I see what you mean.
I agree with what you say regarding confirmation bias but then how do you separate that from what is considered the scientific consensus? What I mean is that Newton's Law is not scientifically accurate anymore (it's good enough, though) but the fact that it validated what we observed (i.e. gravity) is also confirmation bias.
What I'm getting at is that there is a fine line between confirmation bias and scientific theory. I hope I made sense, lol
Ooh that's a good question, how do you control for confirmation bias in studies?
I'm a bit embarrassed to have to admit that this goes beyond my knowledge. I'm sure there are answers to this, this must be well known in these areas of research. We also know that research itself can be biased too. I'll have to ask friends working on these topics! Thanks for the interesting discussion about this I'll probably live in the future.
On this topic specifically though, that meta analysis that concluded there was a lack of evidence was despite the potential confirmation bias (unless the authors of the meta-analysis where already suspicious about the theory… oh well… one can hope them following the scientific method provides strong enough guarantees. It's not completely bulletproof but it's the most reliable thing we have. I'll ask for sure!).
> but the fact that it validated what we observed (i.e. gravity) is also confirmation bias
Pretty sure that's wrong. The way it works is: we have this equation. It predicts where we expect such stuff to be in X seconds. In X seconds, we check it's indeed there. It's there: actual confirmation, not confirmation bias. That's how you check your hypothesis. Of course the initial hypothesis comes from intuition… formed by observing the world. Enough confirmations makes your model more reliable, and is the thing that will be used until a counter example shows its nose and a better model is found. Even then, the model can still be used for cases where we know it does the job; Newton's model is simpler to use than Einstein's so we keep using it.
I guess if you have a solid enough hypothesis, it also works like this in human sciences.
> Pretty sure that's wrong. The way it works is: we have this equation. It predicts where we expect such stuff to be in X seconds. In X seconds, we check it's indeed there. It's there: actual confirmation, not confirmation bias.
Exactly. My point is that since Einstein's theory, we know that Newton's Law is incomplete. Therefore proving that it was confirmation bias (i.e. that our equations just confirmed what we observed). Since we observed black holes, we knew that Newton's was incomplete as it couldn't fully explain their behaviors.
> i.e. that our equations just confirmed what we observed
No, no, it's the opposite, and it's key! What we had been observing kept matching what the equations gave us "so far". Without cherry-picking, or refusing to see the cases where the model doesn't apply (consciously or not), which would have been confirmation bias.
We did, in fact, question the model as soon as we noticed it didn't apply.
Confirmation bias implies "cognitive blinkers", I don't think this happened in this Newton vs Einstein stuff.
But I agree the confirmation bias risk is not very far away. It's an issue in the general population, it's also likely a big issue in research.
2 replies →