← Back to context

Comment by SaberTail

8 days ago

This is one reason science has adopted the notion of a "blind" analysis in which the techniques used to analyze the data are developed without looking at the data. This makes it harder to tweak an analysis to discover what you're hoping to discover.

Usually blind analyses are developed with a mixture of simulated data and known "null" data. For example, if you were developing a blind analysis to look for Vulcan in telescope data, you might create an artificial telescope that renders simulated planets and use that to refine and characterize whatever imaging techniques you're using, and also look apply those techniques to data from real telescopes pointed far away from the sun to make sure they're not finding false positives.

In particle physics, a half century ago, before blind analyses were common, there were a lot of particle "discoveries" resulting from seeing some anomaly in the data, then introducing more and more cuts to refine the data until it was a huge statistically significant peak. But then these would disappear as soon as more data appeared, because ultimately what they'd found was statistical noise.

At least some of the evidence for the Higgs boson comes from blind analyses on LHC data.