Comment by kelseyfrog

I agree. It's not the best metaphor.

I played around with various metaphors but most of them felt various degrees of worse. The idea of relaxing priors and then doing an evidence-based update while thinking it's genuinely a single update is a difficult thing to capture metaphorically.

Happy to hear better suggestions.

EDIT: Maybe something more like this:

Picture your belief as a shotgun aimed at the truth:

    Aim direction = your best current guess.

    Spread = your precision.

    Evidence = the pull that says "turn this much" and "widen/narrow this much."

The correct move is one clean Bayesian shot.

Hold your aim where it is. Evidence arrives. Rotate and resize the spread in one simultaneous posterior jump determined by the actual likelihood ratio in front of you.

The stupid move? The move that Rationalists love to disguise as humility? It's to first relax your spread "to be open-minded," and then apply the update. You've just secretly told the math, "Give this evidence more weight than it deserves." And then you wonder why you keep overshooting, drifting into confident nonsense.

If you think your prior is overconfident, that is itself evidence. Evidence about your meta-level epistemic reliability. Feed it into the update properly. Do not amputate it ahead of time because "priors are bias." Bias is bad, yes, but closing your eyes and spinning around with shotgun in hand ie: double updating is not an effective method at removing bias.