Comment by flaminHotSpeedo

1 day ago

> Sometimes you need to flip a switch

If anyone needs to flip a global switch, and can't convince their leadership to allocate the resources to do it safely, engineering culture is dead, at least locally to that system.

But I'm not convinced lack of headcount was the problem here, the incident report makes it sound like there's an established pattern for feature flagging even for global changes like this.

Putting aside the fact that this team seems unperturbed by global deployments and all the other scary things, high impact changes should use every mechanism available to shrink fault containers. It would be inexcusable to roll this change out without tha feature flag mechanism if this were a regional roll out.

Skipping the feature flag when this is global is simply incomprehensible. It goes beyond headcount, it should never have been considered in the first place.