Comment by kqr
1 day ago
This has been known ever since the beginning of frequentist hypothesis testing. Fisher warned us not to place too much emphasis on the p-value he asked us to calculate, specifically because it is mainly a measure of sample size, not clinical significance.
Yes the whole thing has been a bit of a tragedy IMO. A minor tragedy all things considered, but still one nonetheless.
One interesting thing to keep in mind is that Ronald Fisher did most of his work before the publication of Kolmogorov's probability axioms (1933). There's a real sense in which the statistics used in social sciences diverged from mathematics before the rise of modern statistics.
So there's a lot of tradition going back to the 19th century that's misguided, wrong, or maybe just not best practice.