Comment by spider-mario
4 hours ago
If you just meant it in the sense that “if I pick one interval at random and don’t look at it, it will (future) have an n% chance of containing the true parameter” then sure. It’s a pre-data statement (“I have an n% chance of sampling data that will happen to generate an interval that contains the parameter”).
But once you have picked one, and you know its bounds (say, [12.1471, 13.8264]), then it’s fallacious to make the post-data reasoning that “because it was picked at random from the set of 90% confidence intervals, it has a 90% chance of containing the true parameter”.
Again, it’s like with medical tests. If a test has 90% sensitivity and 90% specificity, it has 90% accuracy (it will, in 90% of cases, produce a result that matches disease status) – a pre-data statement on the test result (/ on the confidence interval that we will compute). But it does not follow that, if you screen an asymptomatic patient with low prior probability of disease and get a positive result, they have a 90% chance of having the disease – a post-data statement on disease status, given the test result (/ on where the parameter lies, given the interval).
> The conclusion they come to is "it is possible to do better in the individual case by taking into account evidence from the sample that the confidence interval method throws away". That means the confidence interval is inefficient, not incorrect.
It means you know in the individual case that the specific confidence interval does not in fact have a 90% chance of containing the parameter.
I can see the analogy here but a test like that being binary throws things off and that's also super asymmetrical error. Weren't we sampling a gaussian?
It’s the same principle. The “trivial interval” from the first link (the one you called pedantry), which is applicable to a Gaussian and tweakable to have different coverage than 50%, clearly shows that being an n% confidence interval, on its own, doesn’t guarantee the ability to directly translate that n% into post-data inferences. Likewise, in the truncated exponential example, when you compute that confidence interval of [12.1471, 13.8264], you know that the probability that it contains θ is 0%. In the Cauchy example, when you compute [-2.31, 10.31] as your 90% CI from the two samples 3 and 5, you can then compute that it actually has a >99% chance of containing θ (table I).