Comment by chrisfosterelli
15 hours ago
Health metrics are absolutely tarnished by a lack of proper context. Unsurprisingly, it turns out that you can't reliably take a concept as broad as health and reduce it to a number. We see the same arguments over and over with body fat percentages, vo2 max estimates, BMI, lactate thresholds, resting heart rate, HRV, and more. These are all useful metrics, but it's important to consider them in the proper context that each of them deserve.
This article gave an LLM a bunch of health metrics and then asked it to reduce it to a single score, didn't tell us any of the actual metric values, and then compared that to a doctor's opinion. Why anyone would expect these to align is beyond my understanding.
The most obvious thing that jumps out to me is that I've noticed doctors generally, for better or worse, consider "health" much differently than the fitness community does. It's different toolsets and different goals. If this person's VO2 max estimate was under 30, that's objectively a poor VO2 max by most standards, and an LLM trained on the internet's entire repository of fitness discussion is likely going to give this person a bad score in terms of cardio fitness. But a doctor who sees a person come in who isn't complaining about anything in particular, moves around fine, doesn't have risk factors like age or family history, and has good metrics on a blood test is probably going to say they're in fine cardio health regardless of what their wearable says.
I'd go so far to say this is probably the case for most people. Your average person is in really poor fitness-shape but just fine health-shape.
Many of those metrics are population or sampling measures and are confounded by many factors at an individual level. The most notorious of which is BMI; it is practically a category error to infer someone's health or risk by individual BMI, and yet doing so remains widespread amongst people that are supposed to know better.
Instrumentation and testing become primarily useful at an individual level to explain or investigate someone's disease or disorder, or to screen for major risk factors, and the hazards and consequences of unnecessary testing outweigh the benefits in all but a few cases. For which your GP and/or government will (or should) routinely screen those at actual risk, which is why I pooped in a jar last week and mailed it.
An athlete chasing an ever-better VO2max or FTP hasn't necessarily got it wrong, however. We can say something like, "Bjorn Daehlie’s results are explained by extraordinary VO2max", with an implication that you should go get results some other way because you're not a five-sigma outlier. But at the pointy end of elite sport, there's a clear correlation between marginal improvement of certain measures and competitive outcomes, and if you don't think the difference of 0.01sec between first and third matters then you've never stood on a podium. Or worse, next to one. When mistakes are made and performance deteriorates, it's often due to chasing the wrong metric(s) for the athlete at hand, generally a failure of coaching.
> The most notorious of which is BMI; it is practically a category error to infer someone's health or risk by individual BMI, and yet doing so remains widespread amongst people that are supposed to know better.
BMI works fine for people who aren't very muscular, which is the great majority of people. Waist to height ratio might be more informative for people with higher muscle mass.
My understanding is that it doesnt even do that, because it creates false negatives for the so called skinny fat body type: significant visceral fat mass, which is what we are concerned about, but not much muscle or peripheral fat mass, thereby not being flagged by BMI screens, even though they are at risk.
As a person who has been told I'm "morbidly obese" for decades now, I will say that doctors at almost every level look at your chart not you. I've been told time and time again that until I get my weight under control, my health will suffer.
I'm 5'8" and weigh on average 210lbs. My BMI isn't even morbidly obese, it is 31, which is just "regular" obese, but on top of that, a DEXA scan shows that I am actually only 25% body fat, with only 1lb of visceral fat.
Doctor's don't care about that, they see on the Epic chart that my BMI is > 30 and have to tell me some spiel about a healthier lifestyle so they check check off a checkbox and continue to the next screen.
4 replies →
I dunno, basing life decisions off a metric that has a fudge factor built into it to make the regression work feels sub-optimal to me.
1 reply →
> But a doctor who sees a person come in who isn't complaining about anything in particular, moves around fine, doesn't have risk factors like age or family history, and has good metrics on a blood test is probably going to say they're in fine cardio health regardless of what their wearable says.
This is true of many metrics and even lab results. Good doctors will counsel you and tell you that the lab results are just one metric and one input. The body acclimates to its current conditions over time, and quite often achieves homeostasis.
My grandma was living for years with an SpO2 in the 90-95% range as measured by pulse oximetry, but this was just one metric measured with one method. It doesn't mean her blood oxygen was actually repeatedly dropping, it just meant that her body wasn't particularly suited to pulse oximetry.
It doesn't help when doctors are often unaware of outliers affecting the test results. E.g. I've had a number of doctors freak out over my eGFR (kidney function) test results because the default test they use is affected by body mass and diet, and made even worse by e.g. preworkout supplements with creatine. None of my doctors have been aware of this, and I've had to explain it to them.
I've not seen evidence that creatine actually has significant impact on eGFR. Anecdotally, mine does not budge even on 5g a day. Meta-analysis show minimal impact, e.g. https://pmc.ncbi.nlm.nih.gov/articles/PMC12590749/
Muscle mass obviously does, though. cystatin c is a better market if your body composition differs from the "average"
4 replies →
>This article gave an LLM a bunch of health metrics and then asked it to reduce it to a single score, didn't tell us any of the actual metric values, and then compared that to a doctor's opinion. Why anyone would expect these to align is beyond my understanding.
This gets to one of LLMs' core weaknesses, they blindly respond to your requests and rarely push back against the premise of it.
Measuring metrics is easy, it's the algorithm on the backend that matters.
There's a reason why Oura rings are expensive and it's not the hardware - you can get similar stuff for 50€ on Aliexpress.
But none of them predicted my Covid infection days in advance. Oura did.
A device like the Apple Watch that's on you 24/7 is good with TRENDS, not absolute measurements. It can tell you if your heart rate, blood oxygen or something else is more or less than before, statistically. For absolute measurements it's OK, but not exact.
And from that we can make educated guesses on whether a visit to a doctor is necessary.
> But none of them predicted my Covid infection days in advance. Oura did.
It actually warned you, or retrospectively looking at the metrics you could see that there was a pattern in advance of symptoms? (If the latter, same here with my Garmin watch - precipitous HRV decline in the 7 days before symptoms. But no actual warning.)
It actually told me, they've been doing this for a while: https://ouraring.com/blog/early-covid-symptoms/
Of course it didn't tell me "you have COVID19-B variant C" - but it did tell me I'm probably sick and should seek care.
I'm curious how the ring detected it in advance? I also discovered my Covid when I looked at my Garmin watch and my resting heart rate was 100, until then I had thought I had too much sun that day.
Some of the metrics were out of whack, I think my average body temp was up along with my resting heart rate both asleep and awake.
It somehow takes all that and gave me a "you might be sick" notification.
2 replies →
>I'd go so far to say this is probably the case for most people. Your average person is in really poor fitness-shape but just fine health-shape.
Modern medicine has failed to move into the era of subtlety and small problems and many people suffer as a result. Fitness nerds and general non-scientists fill the gap poorly so we get a ton of guessing and anecdotal evidence and likely a whole lot of bad advice.
Doctors won't say there's a problem until you're SICK and usually pretty late in the process when there's not a lot of room to make improvements.
At the same time, doctors won't do anything if you're 5% off optimal, but they'll happily give you a medicine that improves one symptom that's 50% off optimal that comes along with 10 side effects. Although unless you're dying or have something really straightforward wrong with you, doctors don't do much at all besides giving you a sedative and or a stimulant.
Doctors don't know what to do with small problems because they're barely studied and the people who DO try to do something don't do it scientifically.
A worthwhile book to read on this topic is Outlive by Peter Attia (MD). The core premise is that American healthcare focuses far too much on treating problems after they’re extremely severe. It is would be cheaper and healthier to invest more into conservative & preventative care, trying to prevent or minimize problems early in life before they become incredibly dangerous and expensive/difficult/impossible to treat.
I have a close friend who works in conservative care, and it’s astonishing what they see. For example, someone went to a number of specialists and doctors about a throat condition where they really struggled swallowing. They even had to swallow a radioactive pill to do some kind of imaging. Unnecessary exposure, and an expensive process to go through, and ultimately went exactly nowhere.
Meanwhile, it was a simple musculoskeletal issue which my friend was able to resolve in a single visit with absolutely no risk to the patient.
Medical schools need to stop producing MDs who reach for pills as the first line of defense without trying to root cause issues. Do you really need addictive pain killers, or maybe some PT, exercise, massage, etc. to help resolve your pain.
> Doctors won't say there's a problem until you're SICK and usually pretty late in the process when there's not a lot of room to make improvements.
As someone who is fit and active,in their 60s with zero obvious symptoms, but is nonetheless on cholesterol and blood pressure medication, this isn't true (in the UK, at least)
One of the things the NHS does surprisingly well, and is only really possible because it's a completely vertically integrated system, is population-level preventative medicine. Distributing insulin and salbutamol. Screening for various sorts of cancer. Cholesterol and BP checks. Encouraging people to stop smoking.
It’s not medicine. It’s healthcare system. Doctor isn’t paid enough to go thoroughly through the complaint and dig deeper. In Germany you get 5 minutes diagnose and that’s all from health insurance. And this from the better doctor. For normal one diagnose comes from 2 minutes interaction. Believing that the diagnose is right is very naive.
It’s also cultural. Most American doctors don’t bother to tell people if they are overweight and out of shape. It’s not something their customers reward.
Maybe I'm not getting you right, but IMO it hasn't? I, as a customer/patient, just don't weekly converse with my MD about small issues, and frankly, they have better things to do, for example treating sick people.
Instead I use the health benefits programs of my health care insurer. My insurer has an interest in prevention, so I can get consulting for free (or very low fees), and even kickbacks if I regularly participate in fitness courses and maintain my yearly check-up routine. Now, I live in Germany and it probably is different in other countries, but it just makes economic sense from the insurer's point of view so that I would be surprised if it were very different elsewhere.
I think one of the major problems is that biologists/scientists cannot legally treat people. Physicians take their studies and have monopolistic treatment powers over them.
I think this creates a huge knowledge gap.
The problem is that the product itself invites the wrong expectation