No. They are sort of good at reading a list of symptoms and having a decent chance of coming up with whatever their training set thinks is a likely diagnosis. They seem to be rather erratic, though, and they also make stuff up that would be trivially rejected by anyone with actual understanding of medicine (and by the same LLM if you quiz it harder!). They also seem to suffer from what I believe is known as the “reverse problem”: if the training set contains “condition A causes symptoms B and C” but not “if you have symptoms B and C then condition A should be considered”, then LLMs seem very likely to miss the diagnosis. Interestingly, real doctors seem to have the same problem, and I suspect that medicine is full of supposedly rare conditions that are actually not so rare but that most doctors are incapable of diagnosing.
No. They are sort of good at reading a list of symptoms and having a decent chance of coming up with whatever their training set thinks is a likely diagnosis. They seem to be rather erratic, though, and they also make stuff up that would be trivially rejected by anyone with actual understanding of medicine (and by the same LLM if you quiz it harder!). They also seem to suffer from what I believe is known as the “reverse problem”: if the training set contains “condition A causes symptoms B and C” but not “if you have symptoms B and C then condition A should be considered”, then LLMs seem very likely to miss the diagnosis. Interestingly, real doctors seem to have the same problem, and I suspect that medicine is full of supposedly rare conditions that are actually not so rare but that most doctors are incapable of diagnosing.