Comment by koliber

Automation, speed, and efficiency has a risk which rarely acknowledged.

I phrase it as "the faster you run, the more painful the fall."

I find this to be universally applicable, from the most literal situations through the most abstract ones.

If I ride my bike, and I hit a bump, I can expect to fall, and perhaps break a bone or dislocate a shoulder. The chance of a very serious injury is relatively low. If I drive my car at 100mph and I hit a bump, there is a good chance there will be a fatality.

Same thing applies to any system.

The more cumbersome, slow, and manual a system, the less of a chance that a systematic and far-reaching failure will happen. The scenario in this article would not have been possible if this was a manual process.

I am not a luddite. Quite to the contrary, I value progress. However, people dismiss black-swam events for large automated systems far too often. Even though the risk of any such event is small, once it happens the damage is extensive. Circuit breakers and manual intervention overrides need to be designed from the get-go. Unfortunately, they are only added after something bad happens.