← Back to context

Comment by andrewflnr

6 hours ago

Sometimes the "system" consists of identifying and removing or bypassing unreliable components.

If you could tell ahead of time which components would be unreliable, that would be a great idea. For the most part, we only discover the component was unreliable after it's caused trouble. Removing the component at that time gets you at best a very short-lived improvement to the system because it'll happen again and again with other components.

If the component removed is a human, the situation is worse. Then you're creating an incentive to hide mistakes and assign blame to others. This makes it even harder to improve the system properly, because then the information you need to improve the system no longer gets to you undistorted.

That's why we have to design systems that don't assume components are perfectly reliable, and ensure any human components that we're not out to get them, we're out to create an environment where they will find it hard to make mistakes.

As a general rule, systems that scapegoat people never fix the underlying root cause.