← Back to context

Comment by apathy

9 years ago

Of course you could have an intersection between recognizable images and random noise. The odds of this occurring with any regularity are infinitesimal, hence the design of experiments. (Recall that there is nothing like a mathematical proof in the physical world -- at a molecular level, some water molecules are moving upstream at any given moment, but by reaching into a stream or tossing some objects into the flow you are taking a large enough sample to determine where most of them are going).

Since you're not going to get proof one way or another, all a well designed and experiment can do is give you evidence. This happens to be more valuable than just about anything else that science has come up with, but it isn't proof.

Which is why the gold standard for a result is replication in a large sample. I could have this very page generated by convolving couple of high entropy random streams. Is it likely to happen repeatedly? Not if the generator is any good. Same principle for randomized trials. You can end up with unbalanced arms (I'm proofing a manuscript where we had exactly this problem). But it's unlikely that they'll be consistently unbalanced across trials with sufficient sample sizes.