Comment by nonbel
6 years ago
A model of whatever process you think generated the data.
EDIT:
I guess I should say that the concept of testing a "null model" without interpreting the fit relative to other models is wrong to begin with. You need to use Bayes' rule and determine:
p(H[0]|D) = p(H[0])p(D|H[0])/sum(p(H[0:n])*p(D|H[0:n]))
Lots of stuff wrong with what has been standard stats for the last 70 years, it literally amounts to stringing together a bunch of fallacies and makes no sense at all.
Thanks for the response. Do you know of any you good blog posts or articles that dive into this a bit more? It looks very interesting.
This is the best description of the main problem (testing your own vs some default hypothesis) I have seen:
Paul E. Meehl, "Theory-Testing in Psychology and Physics: A Methodological Paradox," Philosophy of Science 34, no. 2 (Jun., 1967): 103-115. https://doi.org/10.1086/288135
Free download here: www.fisme.science.uu.nl/staff/christianb/downloads/meehl1967.pdf
Andrew Gelman (andrewgelman.com) has a great blog that often touches on this issue.
Thanks a bunch for sharing, I appreciate it. I'll add these resources to my reading list. I may also pass it along to my brother who getting a graduate degree in psych :)
https://statmodeling.stat.columbia.edu/2019/03/20/retire-sta...