Comment by geysersam
1 year ago
Extremely shoddy story. People back in the day (working in agriculture) had to perform tons of complex tasks. Obviously they were able to reason.
It's clearly only someone quite far removed from any kind of practical work who could become convinced people who don't immediately answer the expected answer to test questions have no ability to reason.
And yet, that's still the state-of-the-art in psychology.
Circa 1990, good ol' Simon Baron-Cohen observed that autistic children answered certain questions (intended to test empathy) in a consistently unusual way, and he decided that meant autistic people had no theory-of-mind. Never mind that the questions were ambiguous, and the scenarios were underspecified. It wasn't until 2012 that somebody (Damian Milton) managed to get the obvious alternative considered by academia. The "no ToM" theory is still implicitly assumed by some new research papers, despite there being no reason to prefer it over the "double-empathy problem" hypothesis.
These seem like really easy kinds of tests to repro or re-examine? How is there a 22 year gap in this? Or is this perhaps mostly a question of not there being enough examination of the experimentation protocol, and so the idea remains despite the underlying experiment being iffy.
Most scientific research involves studying some aspect of the world as systematically as possible. Psychology has the added issue that the experimenters are also studying themselves. This is fairly unique among sciences, so we don't really have any protocols to deal with that: there's double-blinding, but that doesn't help you to analyse your results, or to decide what experiments to run to begin with.
That's one theory. Another theory is that it's just Simon Baron-Cohen: pretty much all the autism research he's done, even the biological theories, show the same "experiment does not actually test theory" issue. I'll illustrate what I mean:
⸻ ⁂ ⸻
Autism diagnosis is performed, basically, by going through a questionnaire that asks about social and play behaviour in childhood. Simon Baron-Cohen noticed that most people diagnosed with autism were male, interested in sciencey things like engineering and maths, and behaved weirdly in social situations, so he set out to explain this.
Building on his mind-blindness (lack-of-ToM) theory, which had been confirmed to his satisfaction, he observed that engineering is a "systemising" activity, and social stuff is an "empathising" thing. Obviously, systemising is for boys and empathising is for girls, so autism must be a condition of brains being too male (dubbed the "extreme male brain" theory).
To test this, he (and his colleagues: "Simon Baron-Cohen" is a synecdoche, since this was all a team effort) came up with a questionnaire to measure the Systemising Quotient, and a questionnaire to measure the Empathising Quotient. They found the expected association between EQ, SQ, and the Autism Quotient score from that first questionnaire (AQ), thus proving they were measuring what they thought they were measuring. (Just like IQ!)
While this proves the empathising–systemising theory, it doesn't quite prove the extreme male brain theory. We know that autism is a developmental condition, and we know that testosterone is the boy juice, so the "extreme male brain" theory predicts that when we measure higher-than-average foetal testosterone (FT) levels, we end up with autistic kids, and when we don't, we don't. Several studies that study amniotic fluid (where Simon Baron-Cohen is second-author) show that FT levels are positively correlated with SQ and negatively-correlated with EQ, and that's pretty slam-dunk. We've found the cause of autism! Hooray!
⸻ ⁂ ⸻
From the way I've explained it, it should be obvious where the issues with his research are. And maybe I'm being too harsh on Simon Baron-Cohen: he was on that paper that actually measured foetal oestrogen levels (also found to be elevated: guess it's not "extreme male brain" after all), and while some of his critics had an intuition that something was wrong with his conclusions, I can't find evidence that anyone in academia actually identified the problems with his work – not until the autistic autism researchers came onto the scene.
The fact remains that he got a knighthood out of this, and approximately none of what he's "researched" is correct. (Excluding some lower-profile work, like demographic studies https://doi.org/10.1192/bjp.2022.21 and https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9314022/, for which he was last author. I'm not sure what last author means in this field, but this kind of work is very important.)
The worst thing is, he's a good sport about it when other people falsify, bit-by-bit, his life's work. It seems like he was (and still is) actually trying his best to do science. I don't think we can safely treat Simon Baron-Cohen as a weird outlier: the problem is with psychology-as-practised-in-academia, not with Simon Baron-Cohen.
(We haven't fixed the questionnaires, by the way. We do have questionnaires that work better – some are listed on https://embrace-autism.com/ –, but they aren't used. Instead, there are protocols to try to work around the fact the used questionnaires are asking the wrong questions. https://www.autism.org.uk/advice-and-guidance/what-is-autism... goes into some detail.)
1 reply →