Comment by highfrequency
2 years ago
> The prebiotic improves cognition (factor score versus placebo (β = −0.482; 95% CI,−0.813, −0.141; p = 0.014))
Always difficult to interpret the effect size of medical studies. Can an expert chime in here? What does this translate to in percentage terms, or number of standard deviations? How can we tell if this is a clinically meaningful delta?
More broadly - why the obsession in medical papers with statistical significance rather than actual, interpretable effect size?
I'll chime in:
(β = −0.482; 95% CI, −0.813, −0.141; p = 0.014): The "β" represents the change in the factor score due to the prebiotic. A negative number here (-0.482) likely means that the prebiotic reduced some kind of negative score associated with poorer cognition, thus improving cognitive function. The "95% CI" stands for the 95% confidence interval, which is a range of values (-0.813 to -0.141) that likely contains the true effect size. The smaller this range, the more precise the estimate. The "p-value" (p = 0.014) tells us the probability that the observed results could have occurred by chance. A p-value of 0.014 means there's a 1.4% chance the results are due to random variation, which is low, so the results are considered statistically significant.
In simpler terms, that study found the prebiotic to significantly improve mental functions compared to not taking anything, with a high level of confidence that this result is not due to chance.
HOWEVER, when you look at the size of the study, it's unimpressive. A larger RCT will be needed. Also, it was a 'remote' study, so they could not measure muscle mass. Also, analyzing stool samples has serious limitations, as there are quite a few species that cannot be cultured (or identified).
That confidence interval seems large, no?
In such a limited RCT, yeah. It does.
Not expert in medical science.
Here's my understanding β is coefficient (slope) for the linear mixed model and 95% CI is the confidence interval for the coefficient. There are no effect sizes given, my guess is it's harder to calculate for linear mixed models when models gets complicated. It's not easy/useful to interpret from coefficient here.
Rather see Table 3 for mean and SD. The mean(SD) change for cognition for Treatment group was 0.40 (0.83) and for placebo was 0.19 (0.70). Linear mixed model showed this was significant when other factors held constant.
Not an expert, but see table 3. They use CANTAB as the test of cognition. Scores range from 0 (worst) to 1 (best). β = the standardized beta coefficient. You do a linear regression, where the cognition score is the dependent variable, and prebiotic status is an independent variable. The values are standardized first (x - mean)/(standard deviation). β is the slope of the regression line. They use the prebiotic group as the reference group.