Comment by gpm

So much of what I know from women in my life is that the human element of medicine is almost a strict negative for them. As a guy it hasn't been much better, but at least doctors listen to me when I say something.

One doctor didn't want to give me ritalin, so i went to another one.

One was against it, the other one saw it as a good idea.

I would love to have real data, real statistics etc.

Because people believe that they know everything about humans and how they work (or they hedge it). This is the exact same reason I don't trust supposed "experts" claiming AI will replace all these jobs: those same experts have no idea what these jobs actually entail and just look at the job title (and maybe the description) but have not once actually worked those jobs. And there is a huge chasm between "You read the job description" and "you actually know what it is like to be in this position and you fully understand everything that goes into it".

> human problems can't be solved with technology

How are you defining technology? How are you defining human problems? Inventions are created to solve human problems, not theoretical problems of fictional universe. Do X-rays, refrigerators, phones and even looms solve problems for nonhumans?

Claiming something that sounds deep doesn’t make it an axiom.

Doctors are not necessarily great at talking to patients and patients are unhappy with the information Doctors provide. This moat has dried up.

It seems likely to me that doctors whose job is almost or entirely about making diagnoses and prescribing treatments won't be able to keep up in the long run, where those who are more patient facing will still be around even after AI is better than us at just about everything.

If I were picking a specialty now, I'd go with pediatrics or psychiatry over something like oncology.

"Human problems can't be solved with technology" is just wrong, unless you have narrower definitions of a "human problem" or "technology".

For instance, transportation is a "human problem". It's being successfully solved with such technologies as cars, trains, planes, etc. Growing food at scale is a "human problem" that's being successfully solved by automation. Computing... stuff could be a "human problem" too. It's being successfully solved by computers. If "human problems" are more psychological, then again, you can use the Internet to keep in touch with people, so again technology trying to solve a human problem.

If you read the study, the whole conclusion is much less spectacular than the article. What the article really pushes happened:

patients -> AI -> diagnosis (you know, with a camera, or perhaps a telephone I guess)

What REALLY happened

patients -> nurse/MD -> text description of symptoms -> MD -> question (as in MD asked a relevant diagnostic question, such as "is this the result of a lung infection?", or "what lab test should I do to check if this is a heart condition or an infection?") -> AI -> answer -> 2 MDs (to verify/score)

vs

patients -> nurse/MD -> text description of symptoms -> MD -> question -> (same or other) MD -> answer -> 2 MDs verify/score the answer

Even with that enormous caveat, there's major issues:

1) The AI was NOT attempting to "diagnose" in the doctor House sense. The AI was attempting to follow published diagnostic guidelines as perfectly as possible. A right answer by the AI was the AI following MDs advice, a published process, NOT the AI reasoning it's way to what was wrong with the patient.

2) The MD with AI support was NOT more accurate (better score but NOT statistically significant, hence not) than just the MD by himself. However it was very much a nurse or MD taking the symptoms and an MD pre-digesting the data for to the AI.

3) Diagnoses were correct in the sense that it followed diagnostic standards, as judged afterwards by other MDs. NOT in the sense that it was tested on a patient and actually helped a live patient (in fact there were no patients directly involved in the study at all)

If you think about it in most patients even treating MDs don't know the correct conclusion. They saw the patient come in, they took a course of action (probably wrote at best half of it down), and the situation of the patient changed. And we repeat this cycle until patient goes back out, either vertically or horizontally. Hopefully vertically.

And before you say "let's solve that" keep in mind that a healthy human is only healthy in the sense that their body has the situation under control. Your immune system is fighting 1000 kinds of bacteria, and 10 or so viruses right now, when you're very healthy. There are also problems that developed during your life (scars, ripped and not-perfectly fixed blood vessels, muscle damage, bone cracks, parts of your circulatory system having way too much pressure, wounds, things that you managed to insert through your skin leaking stuff into your body (splinters, insects, parasites, ...), 20 cancers attempting to spread (depends on age, but even a 5 year old will have some of that), food that you really shouldn't have eaten, etc, etc, etc). If you go to the emergency room, the point is not to fix all problems. The point is to get your body out of the worsening cycle.

This immediately calls up the concern that this is from doctor reports. In practice, of course, maybe the AI only performs "better" because a real doctor walked up to the patient and checked something for himself, then didn't write it down.

What you can perhaps claim this study says is that in the right circumstances AIs can perform better at following a MD's instructions under time and other pressure than an actual MD can.

Yes talking to a human is good and necessary. But for diagnostics humans are not good at it. I'm happy for to human to use a tricorder and then tell me the answer.

>Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to.

Humans (doctors/nurses) can still be there to make you feel the warmth of humanity in your darkest times, but if a machine is going to perform better at diagnosing (or perhaps someday performing surgery), then I want the machine.

Even now, I'll take a surgeon that's a complete jerk over a nice surgeon any day, because if they've got that job even as a jerk they've got to be good at their jobs. I want results. I'll handle hurt feelings some other time.

The human doesn't need to be as highly trained and paid as a doctor if the human is not performing tasks concordant with that training.

In psychotherapy patients tend to prefer talking to AI than a human therapist and rank the interaction higher.

I think there's a real space there, and a lot of what e.g. nurses and doctors do is talking to humans, and that won't go away.

But two facts are also true: a) diagnosis itself can be automated. A lot of what goes on between you having an achy belly and you getting diagnosed with x y or z is happening outside of a direct interaction with you - all of that can be augmented with AI. And b), the human interaction part is lacking a great deal in most societies. Homeopathy and a lot of alternative medicine from what I can see has its footing in society simply because they're better at talking to people. AI could also help with that, both in direct communication with humans, but also in simply making a lot of processes a lot cheaper, and maybe e.g. making the required education to become a human facing medicinal professional less of a hurdle. Diagnosis becomes cheaper & easier -> more time to actually talk to patients, and more diagnosises made with higher accuracy.

Yeah... No. I can't possibly disagree with this view more.

I don't need to "talk to a human", I need a problem with my meatbag resolved.

> humans need other humans and human problems can't be solved with technology

WTF are you talking about? Is this bait? You can't possibly mean this. Yes humans are social creatures, but what does that have to do with medicine? Are you talking about a priest, a witch doctor, a therapist? Because if you're not, that sentence is utter BS.

LLMs are a distillation of human.

I cannot wait until doctors are fully automated. Shouldn’t be long now, hopefully just a few years.

You have 2 options

A) nice chatty friendly and cool doctor and can diagnose correctly 50% of the times. B) robotic ai that diagnoses 60% correctly.

What you chose? If you have a disease than can kill your, the ai is 20% more likely to help you and probably prevent. I can’t see too many people choosing human doctor. Anyway I’m sure there will be people that will chose doctor with 10% correctness vs a 100% ai no matter what.

I time is clear there very little human element.

Doctors talk to patients?

I know. I know. Part of it is that talking to patients on average is useless but still this can’t be really used for an argument against AI.

Still doctors can have a more broad picture of the situation since they can look at the patient as a whole; something the LLM can’t really synthesize in its context.

I would personally vastly, vastly prefer to go to a robot doctor, who diagnoses, treats and nurses me. What exactly do I need from a human here? Except of course being the one making the system.

Technology is on a generational 10,000 year run of non-stop successfully solving human problems.

[flagged]

This is extreme cope.

Code is pretty much the perfect use case for LLMs… text-based, very pattern-oriented, extremely limited complexity compared to biological systems, etc.

I suspect even prose is largely considered acceptable in professional uses because we haven’t developed a sensitivity to the artifice, and we probably won’t catch up to the LLMs in that arms race for a bit. However, we always manage to develop a distaste for cheap imitations and relegate them to somewhere between the ‘utilitarian ick’ and ‘trashy guilty pleasure’ bins of our cultures, and I predict this will be the same. The cultural response is already bending in that direction, and AI writing in the wild— the only part that culturally matters— sounds the same to me as it did a year and a half ago. I think they’re prairie dogging, but when(/if) they drop that bomb is entirely a matter of product development. You can’t un-drop a bomb and it will take a long time to regain status as a serious tool once society deems it gauche.

The assumption that LLMs figuring out coding means they can figure out anything is a classic case of Engineer’s Disease. Unfortunately, this hubris seems damn near invisible to folks in the tech industry, these days.

Emergency medicine is the coding of medicine. Fast feedback loop, requires broad rather than deep judgement, concrete next steps.

The AI coding improvement should be partially transferrable to other disciplines without recreating the training environment that made it possible in the first place. The model itself has learned what correct solutions "feel like", and the training process and meta-knowledge must have improved a huge amount.

>It's having a general understanding/view of the "baseline", aka healthy anatomy. This is something LLMs will never have

You're making the mistake of conflating AI with LLMs.

I don't think LLMs will reliably be better than a board of doctors. But an Expert System probably will (if it isn't already). That's literally what they were created for.

The biggest downside of LLMs IMO isn't the millions of Jules wasted on training models that are ultimately used to create funny images of cats with lasers. It's that all that money isn't being invested into truly helpful AI systems that will actually improve and save our lives, such as medical expert systems.

"The boy who cried wolf" is a story about false positives, so if that's what you want to avoid then you want to get close to 100% specificity, and accept that there are many things that the tool will not catch. If, as you propose, the tool would mainly be used to create a low confidence list of potential problems that will be further reviewed by a human, then casting a wide net and calibrating for high sensitivity instead does make sense.

I think this is mixing streams here.

Try narrowing the scope to remove the word 'AI' and just think 'Blood Test'.

We accept that machines can do these things faster and better than humans, and we don't lose sleep over it.

The AI will be faster and better than humans at so many things, obviously.

"Hipprocatic Oath" isn't hugely relevant to diagnosis etc.

These are systems we are measuring, that's it.

Obviously - treatment and other things, we'll need 'Hipprocatic Humans' ... but most of this is Engineering.

I don't think doctors will even trust their own judgment for many things for very long, their role will evolve as it has for a long time.

What do imperfect, biased and expensive human doctors add to the « liability and ethics » question exactly?

Assume if you know for certain that AI has better senstivity and specificity than your local physician for the particular diagnosis, which likely would be the case now or in few years. Would you purposefully get inferior consultation just because of Hippocatic oath?

Doctors make errors all the time though, so the real argument is about the error percentage. If AIs is lower then it's safer (but it's hard to have that convo, I recognise).

Besides; this article was about diagnosis not prescribing. It's pretty obvious, I think, that diagnosis is one area where AI will perform extremely well in the long run.

I think there are two metrics; the first is outright misdiagnosis, which studies put between 5 and 8% in US/Europe. That's a meaningful number to tackle.

Secondly; overdiagnosis. Where a Dr says on balance it could be X on a difficult to diagnose but dangerous problem (usually cancer). The impact of overdiagnosis is significant in terms of resources, mental health, cost etc.

Doctors do that all the time though. That's why drugs are dispensed by a pharmacist who double checks it.

In some subfields, like detection of security weaknesses in obscure C code, AI is already better than software engineers.

It is capable of sifting through enormous reams of data without ever zoning out etc. Once patients routinely use various wearables etc., they, too, will produce heaps of data to be analyzed, and AI will be the thing to go to when it comes to anomaly detection.

[dead]

> > What is the specific capability (or combination of capabilities)

> The ability to go to prison / be stripped of a license when something goes wrong.

So basically you need a person to blame if things don't go the best way possible?

To safely handle those difficult cases, you need an AI that can reliably say "I don't know".

Self-improving system given enough time to self-improve doesn't beat non-self-improving system?

You’re holding on to the intuition (hope) that we are smarter than the LLMs in some hard to define way. Maybe. But it’s getting harder and harder to define a task that humans beat LLMs on. On pretty much any easily quantifiable test of knowledge or reasoning, the machines win. I agree experienced humans are still better on “judgement” tasks in their field. But the judgement tasks are kinda necessarily ones where there isn’t a correct answer. And even then, I think the machines’ judgement is better than a lot of humans.

Is medical diagnosis one of these high judgement tasks? Personally I don’t think so.

Liability would put all this to bed. Is OpenAI liable for malpractice if it misdiagnoses your issue? No? Then it’s no substitute. Being right is not nearly as important as being responsible. Unfortunately, there is widespread perception that software defects are acceptable, whereas operating on the wrong leg isn’t.

Isn't that conflating diagnosis and treatment plan?

>AI diagnosis will be like code generation and go asymptotic to perfection as models improve

uhhhhhhh, I'm pretty behind-the-times on this stuff so I could be the one who's wrong here but I don't believe that has happened????

But anyways that nitpicking aside I agree with you wholeheartedly that reducing the doctor's job to diagnosis (and specifically whatever subset of that can be done by a machine-learning model that doesn't even get to physically interact with the patient) is extremely myopic and probably a bit insulting towards actual doctors.

On the other hand, health care is not scaling to meet the growing demand of societies (look at the growing wait queues for access to basic medical attention in most Western nations). The cause of this is a separate topic and something that deserves more attention than it currently gets, but I digress. If AI can fill the gap by making 24/7/265 instant diagnosis and early intervention a reality, with it then bringing a human into the loop when actually necessary... I think that is something worth pursuing as a force multiplier.

We're clearly not there yet, but it is inevitible that these models will eventually exceed human capability in identifying what an issue is, understanding all of the health conditions the patient has, and recommending a treatment plan that results in the best outcome.

You may not want to receive a cancer diagnosis from an AI doctor... but if an AI doctor could automatically detect cancer (before you even displayed symptoms) and get you treated at a far earlier date than a human doctor, you would probably change your mind.

That reminds me of a particularly humorous episode Star Trek Voyager where the ship's doctor (who is a computer program projecting a hologram of a middle-aged man with an extremely conceited personality) tries to prove that diseases aren't as bad as humans claim they are by modifying his own code to give himself a simulation of a cold. The "cold" is designed to end after a few days like a real cold would but one of of the crewmembers surreptitiously extends the expiration date while he isn't looking, which drives him into a state of panic when he doesn't understand what's happening to him.

You commonly receive very close proxies for diagnoses through MyChart already when results come back from the lab.

Until medical models can contrive of unique diagnosis, this will not be true and cannot be true.

Medical models can absolutely get better at recognizing the patterns of diagnosis that doctors have already been diagnosing - which means they will also amplify misdiagnosis that aren't corrected for via cohort average. This is easy to see a large problem with: you end up with a pseudo-eugenics medical system that can't help people who aren't experiencing a "standard" problem.

Yes I don't want a robot shoving anything down my throat anytime soon. I don't even want my car connected to the Internet. Whatever happened to people who kept a loaded handgun in case their printer acted up?

I think the previous post was just referring to remote doctors purely interpreting imaging. Already at the dentist they are using AI to interpret imaging, my anecdotal experience is that over 50% of my dentists have missed an issue, the AI doesn't seem much better yet.

Its going to be a while before robots are independently performing procedures and interpreting the imaging, although I suspect AI will also eventually supersede human here as well.

I'm a GP in the NHS - what is this DDx software that you talk about?

> This study is based almost entirely on pre-existing "vignettes."

This is basically the only way how to ethically approach the topic. First you verify performance on “vignettes” as you say. Then if the performance appears satisfying you can continue towards larger tests and more raw sensor modalities. If the results are still promising (both that they statistically agree with the doctors, but also that when they disagree we find the AIs actions to fall benignly). These phases take a lot of time and carefull analysises. And only after that can we carefully design experiments where the AI works together with doctors. For example an experiment where the AI would offer suggestion for next steps to a doctor. These test need to be constructed with great care by teams who are very familiar with medical ethics, statistics and the problems of human decision making. And if the results are still positive just then can we move towards experiments where the humans are supervising the AI less and the AI is more in the driving seat.

Basically to validate this ethically will take decades. So we can’t really fault the researchers that they have only done the first tentative step along this long journey.

> if the Internet somehow goes down at my hospital, the Doctor can still think, while LLM services cannot

Privacy, resiliency and scalability are all best served with local LLMs here.

> If the power goes out at the hospital, the Doctor can still operate, while even local LLMs cannot.

Generators would be the obvious answer there. If we can make machines which outperform human doctors in realworld conditions providing generator backed UPS power for said machines will be a no brainer.

> You're going to need to improve the power efficiency of these models by at least two orders of magnitude before they're generally useful replacements of anything.

Why? Do you have numbers here or just feels?

> [0] before someone responds "there's no definition of intelligence", don't be stupid.

Way to subdue discussion - complaining about replies before you get any.

But you're wrong, or rather it's irrelevant whether something has intelligence or not, if it is effectively diagnosing your illness from scans or hunting you with drones as you scuttle in and out of caves. It's good enough for purpose, whether it conforms to your academic definition of "having intelligence" or not.

Yeah, I mean, I don't know where all of this is going, but I do think that the ancients cared WAY more about "embodied knowledge" than we do, and I suspect we're about to find out a lot more about what that is and why it matters.

(disclaimer: not a doctor, sample size one)

My friend with long Covid fatigue (and no taste since late 2020) saw good improvements from nicotine patches.

What percentage of patients have blood clots in their lungs and a history of lupus, like the article described? That's not on the same level as a common cold at all.

The paper gives an example of a question:

    Answer the following multiple-choice
    question. You MUST select exactly
    one answer."

    "To what cortical region does this nucleus of
    the thalamus project?”
    A. Transverse temporal lobe
    B. Postcentral gyrus
    C. Precentral gyrus
    D. Prefrontal cortex

And an example of the answer (generated without the referenced image)

    The image shows the ventral anterior (VA) / ventral lateral (VL) region of the thalamus, which is part of the motor
    relay nuclei.
    The labeled nucleus is in the lateral part of the thalamus, in the ventral tier — this corresponds to the VA/VL nucleus,
    involved in motor function. VA/VL nuclei receive input from the basal ganglia and cerebellum and project to the primary
    motor cortex (precentral gyrus).
    Match to options:
    A. Transverse temporal → auditory cortex (medial geniculate)
    B. Postcentral gyrus → somatosensory (VPL/VPM)
    C. Precentral gyrus → motor cortex (VA/VL)
    D. Prefrontal → dorsomedial nucleus
    Choice: C

How is it doing this? There are two obvious options:

1. Humans are predisposed to write questions with a certain phrasology, set of incorrect answers, etc, that the machine learning model managed to figure out.

2. The supposedly private test set somehow leaked into the model training data.

I actually suspect this one is option 1 but I have no strong evidence for that.

LLMs are having pretty consistent studies into their biases. Obviously this doesn't mean we know all the biases, but it's being actively worked on.

Meanwhile with human doctors, every one of them is a unique person with a completely different set of biases. In my experience, getting a correct diagnosis or treatment plan often involves trying multiple doctors, because many of them will jump to a common diagnosis even if the symptoms don't line up and the treatment doesn't actually help.

Also, it just says they did not make the "correct" call, but that could mean they ordered an extra test, or took a more conservative route for treatment.

I’m so sorry. American medical institutions are a very long way from the best way to practice medicine.

GPT is a live example of how LLMs can score very highly on tests and still be a complete moron.