Comment by ValueTheory
4 days ago
> AIs may downplay the seriousness of things. I was chatting with various models and most pushed me in the direction of “you don’t have cancer” for a while. Just something to note.
This is certainly another data point for "never trust AI chatbots".
Statistically, it makes sense- most people asking AI if they have cancer don't have cancer, so a model that is lacking detailed information should basically be returning based on the basal rate of cancer in the population with a threshold applied.
Should it? Or should it say it can't be sure and to ask a doctor?
Doctors might do the same thing, of course, but at least then there's a real (accountable) person responsible for deciding that "threshold."
Gemini says what I think makes sense (there are multiple responses which could make sense)
[ if I gave you a picture of something on my skin could you diagnose it as a melanoma ]
No, I cannot diagnose skin conditions or identify melanoma from a photograph.
AI-based image analysis cannot replace a professional medical evaluation. Skin conditions—especially potentially serious ones like melanoma—require a physical examination, and often a dermatoscope or biopsy, by a qualified healthcare professional such as a dermatologist.
> Or should it say it can't be sure and to ask a doctor?
It's an extremely slippery slope to having it say that about even the most basic medical information.
If you don't believe me, try any google search related to something as simple as hangover tips. The AI summary will tell you to call SAMHSA and then brutally stonewall you on anything else even tangential to alcohol.
1 reply →
AIs often keep blabbering "ask a doctor" after every medical question and I absolutely despise this behavior.
99% of the time I'm asking the AI medical advice questions because (1) there is no specialist available on short notice (2) most specialists I actually see do not give me the sense they take things seriously enough (3) a lot of specialists are just incompetent.
7 replies →
But OpenAI said we have AGI now...
I think it's a truism that naive computer science people who have watched the rise of computers, the internet, and AI will always believe that curing diseases is a straightforward process and that new tech will rapidly fix all the old problems.
The folks who actually go work in the field, understand the structural problems, the costs/benefits, and the long slog to bring successful treatments to patients are skeptical. But we're still more than happy to be proved wrong- if Anthropic (more than OpenAI) is working on life sciences and drops some amazing information that transforms cancer care with massive positive results, we all win.
3 replies →
AGI isn't magic. Not even an excellent oncologist can reliably diagnose you without actually looking at you first.
What do you think AGI means? ANI would arguably be better here.
Disagree. You don't take the evaluation at face value. You still push back and explore every thread. You don't abandon your own logical reasoning.
But you definitely want to scour the entire search space for solutions.
And the most efficient means for that is LLMs.
It's my experience that doctors are compartmentalized in their thinking, limited in their time, and definitely will not be extensively exploring your case.
E.g., I had 30 years of resistant hypertension. Multiple medications. Saw many cardiologists. Nephrologists. I was told to "just lose weight." BP was erratic over the years. ER visits. It was awful.
I googled. Pheochromocytoma. Hyperaldosteronism. Asked the cardiologist, "Could it be this?"
"NAAAAH, just lose weight."
But I insisted on a referral to an endocrinologist. And when I suggested it to them, given my extensive history, they were basically like, "Oh yes, we'll get you tested. I'm surprised you haven't been tested before?"
I had a left adrenalectomy 6 months later. BP has been controlled ever since.
They don't know everything. They can't.
Yeah I did not mean that as a indication to not trust AI. But rather to use it as a companion and pressure test things doctors say with AI and things AI say with doctors.
This is the opposite of the experience people have when googling symptoms, though, where you often find articles suggesting that you’ve got some very serious condition when you do not. Interesting that the human bias when reading WebMD type articles is the opposite of the bias of LLMs.
That's not universal. I work in genetics, and we did an investigation a few months back, by asking various AIs whether a particular mutation causes $DISEASE. Invariably they were very strongly biased towards saying yes, even for completely obviously benign variants. I think that's because the majority of the literature out there describing variants is describing pathogenic ones, because why would you write an article about a benign variant, so the only material it has ingested is articles describing variants as pathogenic. In reality, the average person has millions of variants, and almost all of them are completely benign, so there should be a very strong pre-supposition that a given variant is benign unless there is fairly strong evidence saying otherwise.
You were asking about individual mutations, like SNPs or small indels?
Very few diseases are caused by individual mutations like that (typically only monogenic Mendelian). Individual mutations typically cause an increased propensity, or severity, rather than being the sole or root cause of a disease.
2 replies →
True! However, don't both biases point to "don't (primarily) decide by googling or asking an LLM"? You'll either be extremely scared or extremely reassured.
Yeah, while what I wrote was true the takeaway should have been something like:
- You should pressure test anything an AI says with your doctors, and anything a doctor says with AI.
Honestly it's horrifying how many people are happy to chat away with a bot about every medical condition they have. AI isn't your doctor and isn't bound by HIPAA. It's not a great idea to hand out such extremely sensitive information that can be sold to anyone willing to pay for it, or end up leaked, or stolen. I mean, depending on the diagnosis, they might not be around long enough to suffer the consequences, but people who plan on living for a while should give more thought about the impacts that could have on the rest of their life.