Comment by perpetuallunch
2 days ago
This is completely unsurprising, and this:
But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground
Is an utterly irrelevant comparison. For physical thing we have engineering, and the practical application of the trades and craft, trial and error.
For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that.
Indeed, I suspect the failure rate of, say, new jet engine designs is rather high as well -- those failures just never get reported in a federal repository, unlike RCTs, since they never make it out of the simulator or the prototyping lab. And we have, comparatively, much better computational models of how airplanes fly than how cancer cells mutate. FWIW this clinical stat is far better than Edison's supposed lightbulb-idea failure rate!
Jet engine design is iterative, and the basic principles are well-understood.
Drug design seems a lot more binary. You can find a new pathway, but drugs themselves are fairly simply molecules, and you can't iteratively 'fix bugs' the way you can in an engine or a piece of software.
The engine is a given, it's almost astronomically complicated, you know a lot less about how it works than you'd like to, and you're trying to change how it works while it's running without breaking anything, using tiny rigid parts that have to snap into place correctly and can't be bent to fit.
High failure rates aren't surprising.
This is a pretty outdated view, particularly with respect to biologics, which are some of the most complex structures humans fabricate. Many are in fact quite similar to iterative prototypes of jet engines.
You might have something like Anti–vascular endothelial growth factor therapy, where you have a binding target structure, and iterate the uses and molecular structure around it, or combine it with other structures and binding sites.
You might go from mab to fab, or to bispecific mab using CrossMab IgG architecture, or bispecific fab using dutafab fragment architechture. This analogizes to mixing and matching different jet engine technologies into different platofrms.
VEGF targeting biologics have netted >150 billion dollars to date, and this will only grow faster in the future.
iterative GLP-1 technologies will be much the same, where people are literally iteratively fixing bugs.
The development pipeline for both is littred with failed iterations.
Also, highly related, in the previous paragraph, they say: "We know a lot more about the biology of disease - although God knows, not nearly enough".
Anyone smarter than a 10 year old would not we actually know almost everything about car design. How it works is not at all hidden. We have iterated on pretty much the same thing for 100 years. The current state is about efficiency, materials and manufacturing.
When you have deep knowledge and experience in a field you will get near 100%.
And, moreover, the human body is much, much more complex than any plane or car. We know how cars work and how planes work. For many parts of the human body, we don't know exactly how they work. We have multiple hypothesis, maybe, but we can't say "oh yes this is how serotonin works in the brain". We might know the small scale interactions, but why do SSRIs work in terms of the entire brain? We don't know. Why and how does anesthesia work? We also don't know.
> For physical thing we have engineering, and the practical application of the trades and craft, trial and error.
> For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that.
You could say the same about deep learning. Yet we see improvements every day.
> For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that.
Why do you say that? What's the evidence? We continue to have virtually no clue how to make drugs, per TFA.
Targeted gene therapy, cancer survival rates, trauma care, the advancements in hip and knee replacement, all sorts of surgery, HIV is now a non-issue with the right care. The list goes on.
Drug development is a hard problem because the solution space is poorly constrained: biochemistry is complex and messy, expecting one chemical substance to have narrow positive effects is probably hopeless.
...?
So the claim is that we're out of the wild west because there's a set of things we've made advancements on, despite drug development getting harder and harder?
3 replies →
> Is an utterly irrelevant comparison. For physical thing we have engineering, and the practical application of the trades and craft, trial and error. > > For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that.
I don't think its irrelevant, but a better comparison would be to what vacuum tube development looked like before we understood electrons. There were some very whacky designs and most of them didn't work for crap.
And, yeah, I would argue that PCR shifted us from the alchemy phase of biology to the science phase and now mRNA has shifted us from the science phase of biology to t he engineering phase of biology. We're just getting started.
If someone said, 91% of software projects fail, i would beieve them.