Comment by qsera
1 day ago
>highest quality evidence generation system we could possibly have, which is RCTs.
The "best we have" does not mean it is good enough for the task at hand. Just saying.
Basically what I am saying is that qualifying something by saying "best we have", does not justify its using on its own..
1. I didn't say "the best we have." I said the best we could possibly have. There is no form of evidence generation, real or hypothetical, greater than the randomized controlled trial.
2. That doesn't necessarily make it always "sufficiently good evidence", but the beauty of statistics is we actually can know – quite precisely – whether a given evidence generation method gives us sufficient certainty. When an RCT is used as evidence for approval, it is not "well you did an RCT so I suppose it's fine." Approvals actually are not dependent whatsoever on the evidence generation method. The only thing that matters is whether you prove – to sufficient statistical certainty – the claims you are making. It's very hard to do this without RCTs (again being the greatest evidence generation form that could exist), which is why they tend to be the default. But you can run an RCT that fails to produce certainty, and your drug application will be denied. You can also get approval based on a non-RCT (but again it's hard to do because statistics).
We've had this discussion before and the crux of the issue is that you do not understand statistics while the people who design, run, and evaluate clinical trials do. I highly recommend taking a statistics course or four.
>There is no form of evidence generation, real or hypothetical, greater than the randomized controlled trial.
Wrong. You are lying by omission.
Randomization and controlling are essential. But the version of RCT that we currently employ have problems with control and evaluation. If you have a flaw in control or evaluation, then the whole thing becomes flawed.
So the idea of RCT is good, but we are limited by our ability to control and evaluate.So there can be a better version of RCTs that we currently employ, but with precise control and evaluation.
> but the beauty of statistics is we actually can know
The beauty of statistics is also that it is quite easy to hide falsehoods behind statistics.
There Are Three Kinds of Lies: Lies, Damned Lies, and Statistics
(I am not even going into the fraud problem, which is what we have discussed before, which turned out to be quite useless)