Comment by captainclam
4 hours ago
I also work in drug discovery, and I completely agree; the extent to which the actual causal mechanisms of the efficacy of many drugs is woefully short of some human-appreciable first-principles based explanation. The point is especially undercut by hypothetically suggesting that this drug /does/ in fact pass stage 3 trials, at which point I can't fathom anyone would have a problem advancing it.
Really wish he had chosen a different example; this particular bullet point has been making the rounds on X/twitter to paint Tao as an example of some sort of gatekeeping luddite who would deny the world a post-abundance future in order to maintain the prestige of his particular career path...which is tough because I cannot think of a more responsible steward of our inevitable AI future than Tao at the moment.
The accounts spreading that narrative on X no doubt have their own agenda. Before people start hyperventilating about "AI" conquering human endeavors, remember that the cash-strapped frontier labs are themselves still hiring human "Account Associates" and "Android Engineers" instead of saving those salaries using their own AI capabilities.
https://openai.com/careers/search/
Tao actually pictures the AI as having hacked the FDA, so as to pass the test despite the drug being lethal.
Picking out that one example and misrepresenting it on top of it indeed clearly is driven by an agenda. Or mere stupidity.
The central point in all of this is the misguided idea of "letting AI think for us". An over-generalization feeding into the society-wide problem of self-infantilization.
>to paint Tao as an example of some sort of gatekeeping luddite who would deny the world a post-abundance future in order to maintain the prestige of his particular career path
why is it so hard to accept that a brilliant mathematician can have the same human flaws as anyone else i.e. ego, arrogance, jealousy and competitiveness?
Instead of bemoaning his specific and clumsy example, take the gift of this moment and don’t forget that he’s smart, confident, and usually wrong or completely ignorant outside of his own narrow field.
Don’t let the Gell-Mann Amnesia take you.
I think logic is squarely in his field, and even with simple logic, proposing the importance of second order effects on people who are dying is not tenable. Personally, knowing that he is indeed one of the smartest people around, and definitely much smarter than me, I cannot but be a bit worried that going with this example indicates traces of panic in the face of a shifting world. He’s smart but human, and maybe these weaknesses are not to bemoan but to celebrate in our current context.
One of life’s hardest lessons is that raw intelligence isn’t a superpower, it isn’t a substitute for actually knowing about the topic. The smartest person alive talking about a subject they don’t understand the details of is no better than a confident idiot.
Speaking for myself n I’ll respect his intelligence more when he discovers his limits and learns to act with humility outside of his narrow field.
The conflations about his example ("thought experiment") is twofold, the AI itself confounds the ethical intuition so it is wrong directly compare against what real medicine today does; furthermore, the very admission that real medicine in fact operates through unknown risks (e.g. Jannsen vaccine recall) is different than having a scientific standard that it should not have to be that way at least as minimally as possible.