← Back to context

Comment by cladopa

2 days ago

> How do you write programs when a bug can kill their user?

Engineers had this responsibility for a long time. The Code of Hammurabi talked about what to do when a builder builds a house and it collapses: Put the builder inside a house and make it collapse. Different versions of this method exist today without killing, but ruining the rest of your life anyway. Today if you build a bridge and it collapses, a house and it collapses or it burns, a car or a motorway that provokes accidents.

A plane that falls down, a toxic detergent, a product that is carcinogenic, a nuclear plant meltdown...

In the case of a Pacemaker is actually not that hard, it is a very simple device and you will die if you don't use it for sure, so first patients accepted the risk of dying from the pacemaker malfunction gladly against the alternative of not using it and dying for sure.

There is a methodology for creating things that can't fail, that goes from exhaustive testing to redundancy elements. As an engineer I have studied and applied it. Some times testing can make things ten to twenty times more expensive, like in aviation or medical components.

There is the option of failing often and learning from mistakes. It is by far the cheapest and fastest option. It was used by early aviators, most of them died testing their machines, but we got amazing planes as a result. It was used in WWII by Americans producing in enormous quantities against much better German machines in smaller numbers. It was used by Soviet Russia to compete against the West spending way less money(and sacrificing people) and it is used today by Elon Musk creating rockets. Thanks to automation we don't need to risk lives while testing, only equipment.

Now some people criticise Elon for testing autopilot with people: It can kill people!!, they say. But also not using it kills people when someone has not slept well and needs to go to a job meeting and falls asleep at the wheel and kills herself and some other driver on the road.