Comment by 7952
11 hours ago
And that might work if there is a linear relationship between apparently unnecessary engineering work and deaths. My argument is that such a relationship does not exist, or is not something we can model.
As this is HN I assume you have some understanding of software/IT etc. Do you think a project manager on a massive software project could do the same with security flaws? Reduce the engineering effort by some percentage and get a predictable increase in security issues? And lets say that this project has massive amounts of sunk costs, is hugely important for the livelihood of everyone involved and also classified and closed source. All you have to do to reduce costs is increase data breaches from one to three per year. Easy. But in a complex human-technical system leadership do not have that kind of control authority.
I get your point and would be much more inclined to agree if we were talking about trying to hit a 3x risk increase. But we are talking about huge risk margins here, many OOMs.
My problem with your argument is that as framed it’s a fully generic argument against doing anything; there is always a risk of bad outcomes for any action. What we must do in practice is look at risk/reward and try our best to estimate each.
Data breaches are a bad analogy because you are presenting this as “I get to make a bit more money by lowering security”. A better analogy would be something like colonoscopy; some people will die from cancer if you advise nobody has this procedure. Some people will die from complications if people do get this procedure. How do we as a society choose how many people should die and from what? This is a trolley problem, there is no choice where people don’t die as a result of the decision. The answer is that we must do our best to estimate the risks and minimize them.
This is not what we are doing with nuclear right now. We are simply trying to reduce the risk of nuclear, without making any attempt to model the harms that are being introduced.