Comment by kqr
3 years ago
When people ask about whether the Jones' have children, they do not (most of the time) have an accurate conditional probability of dog ownership given a certain number of children. They are trying to construct a story that seems plausible.
The difference between Bayesian reasoning and narrative fallacy is a coherent evaluation of joint probabilities -- the bit most people skip over.
It's doubtful people have access to population-level base rates either, in which case they would also arrive at the wrong answer, no narrative needed.
But why do you point the finger at narrative instead of a simple lack of data? In the dog example, those questions are absolutely good ones to ask, so long as the data can be found.
One reason to reject this mental opposition between 'Bayesian reasoning' and 'narrative fallacy' is because narratives are essential to coming up with good conditions on which to split the population in the first place.
'I would ask whether they have children, because children like dogs and therefore it's plausible that families with children have a higher rate of dog ownership' is a reasonable narrative that justifies collecting an extra column in your dataset.
Granted, one shouldn't accept that story uncritically without checking against the actual statistics. But narratives themselves are unavoidable even when you're doing statistics correctly!
This is one of those things where it's impossible to prove one approach better or worse than the other. But history has shown (starting with merchant ship insurance in the 1700s going up to Tetlock's superforecaster research more recently) that starting from general base rates gives, on average, more realistic odds than starting from specifics.
You see this all the time in sports betting, too. People overcorrect from the base rate when receiving news. People love a narrative that makes sense to them, but statistically it rarely holds up against statistical reasoning with a very small set of variables. (For more examples, see Meehl's research around clinical vs actuarial judgment.)
Yes, you're correct that a hypothesis is one of those narratives, but if you're operating at that level of scientific abstraction you can ignore my comment. Most people don't, and just roll with the story and forget about the probabilities.