Comment by karmakaze

Not the first cause, but involved. Before reading I expected to see some combination of (1) automation (2) DNS (3) BGP. I didn't expect to see all three and the special automatically disconnect the backbone from the internet with no other way for senior tech staff to get to the backbone, not even a secure dial-up console.

I think the general lesson here is for each thing you automate, assume that it can act in error and have another manual way to do what the automatic action prevents.