> The power analysis used a power criterion of (1 − β) = 0.80, which indicated that a mean cell size of n = 54 was sufficient to replicate significant effects of acetaminophen.
The question is not whether n=114 is "large enough" because that question, by itself, does not make sense. The question is whether n=114 is "large enough for the observed effect." Given that the p-values for the "personal pleasure" and "empathetic feelings" measures are under 0.001, the answer is "yes," potential methodological issues notwithstanding.
Disagree from experience. I find it far more likely that they considered how to skew the p-values low enough to where they become significant. “Personal pleasure” & “Empathetic feelings” do NOT sound like terms which correlate to reproducibility to me.
> “Personal pleasure” & “Empathetic feelings” do NOT sound like terms which correlate to reproducibility to me.
All studies that use terms like that have a section that explains how they are measuring it, how other studies have used the same measure and how that measure itself is tested for reproducibility and accuracy.
I don't think you should sneer at a triple-digit sample size that's justified with a power analysis based on their previous work.
I agree that you'd want to see replications before making any major decisions--these effects are often finnicky--but this seems to me like a reasonable-sized step forward.
Seems like every time now, when I read a medical study, especially psychiatry-related, I'm shocked that it could even be published. I have no training in the field, so nobody would take me seriously and I even doubt myself, but the study designs just seem so poor
I tend to agree. While I think the sample size is fine, the way perceived positivity and perceived pleasure were measured seems pretty subjective in a way I find hard to generalize.
The paper does seem awfully suggestive, though. Worthy of more exploration for sure.
All undergrads from a single university also. Just based on the social development of people as they go through undergrad, I'd guess that the variance in behavioural change before and after treatment is fairly high whether they're treated or not.
Sure, do that. But also:
> The power analysis used a power criterion of (1 − β) = 0.80, which indicated that a mean cell size of n = 54 was sufficient to replicate significant effects of acetaminophen.
The question is not whether n=114 is "large enough" because that question, by itself, does not make sense. The question is whether n=114 is "large enough for the observed effect." Given that the p-values for the "personal pleasure" and "empathetic feelings" measures are under 0.001, the answer is "yes," potential methodological issues notwithstanding.
Disagree from experience. I find it far more likely that they considered how to skew the p-values low enough to where they become significant. “Personal pleasure” & “Empathetic feelings” do NOT sound like terms which correlate to reproducibility to me.
> “Personal pleasure” & “Empathetic feelings” do NOT sound like terms which correlate to reproducibility to me.
All studies that use terms like that have a section that explains how they are measuring it, how other studies have used the same measure and how that measure itself is tested for reproducibility and accuracy.
5 replies →
I don't think you should sneer at a triple-digit sample size that's justified with a power analysis based on their previous work.
I agree that you'd want to see replications before making any major decisions--these effects are often finnicky--but this seems to me like a reasonable-sized step forward.
They also had a prior justification for running this experiment based on known effects of Tylenol on the brain.
Seems like every time now, when I read a medical study, especially psychiatry-related, I'm shocked that it could even be published. I have no training in the field, so nobody would take me seriously and I even doubt myself, but the study designs just seem so poor
I tend to agree. While I think the sample size is fine, the way perceived positivity and perceived pleasure were measured seems pretty subjective in a way I find hard to generalize.
The paper does seem awfully suggestive, though. Worthy of more exploration for sure.
There's been a few over the years.
[1] 2016: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5015806/
[2] 2015: https://journals.sagepub.com/doi/abs/10.1177/095679761557036...
[3] 2012: http://www.psychologicalscience.org/news/releases/broken-hea...
All undergrads from a single university also. Just based on the social development of people as they go through undergrad, I'd guess that the variance in behavioural change before and after treatment is fairly high whether they're treated or not.
Just like a lot of pharma papers need a `... in mice` appended to their titles, a lot of psych papers need `... in undergrads` appended to theirs.
Scientist: Hey, that's odd ...
HN reader: I'll wait for the replication.