Comment by X6S1x6Okd1st
3 years ago
Pretty sure the base rate you'd want to compare against is either placebo given -> remission or simply spontaneous remission. It appears that spontaneous remission is really quite rare.
> [sponanteous remission of cancer] incidence is roughly one in every 60 000–100 000 cancer patients, but the true figure is unknown (2). Spontaneous regression of colon cancer seems to be particularly rare
https://academic.oup.com/jjco/article/45/1/111/888056
So using 0.0016% and 12 patients (which is what the paper the NYT actually links)
For a ~50% chance of seeing one trial with 12 patients have complete remission you'd expect to see ~43k trails.
(1 - 0.000016)^43000 = 0.502577
I wouldn't discount this study based off of those numbers.
If spontaneous remission is super rare it does change the conclusion, yeah.
(Tangentially I think your stats are a little messed up there - you calculated the expected number of people out of 43000 surviving, not the trials returning a false positive. Assuming 0.0016% that chance is astronomically small, so your argument is stronger.)
whoops good point!