Comment by jcgrillo
2 years ago
For some reason this reminded me of the "500mi email" bug [1], maybe a similar level of initial apparent absurdity?
2 years ago
For some reason this reminded me of the "500mi email" bug [1], maybe a similar level of initial apparent absurdity?
The most absurd thing to me about the 500 mile email situation is that sendmail just happily started up and soldiered on after being given a completely alien config file. Could be read as another example of "be liberal in what you accept" going awry, but sendmail's wretched config format is really a volume of war stories all its own...
My favorite example of that was a while ago, "vixie-cron will read a cron stanza from a core dump written to /etc/cron.d" when you could convince it to write a core dump there. The other crons wouldn't touch that, but vixie-cron happily chomped through the core dump for "* * * * * root chmod u+s /tmp/uhoh" etc.
Configuration changes are one of those areas where having some kind of "are you sure? (y/n)" check can really pay off. It wouldn't have helped in this case, because there wasn't really any change management process to speak of, but we haven't fully learned the lesson yet.
Confirmations are mostly useless unless you explicitly spell out the implications of the change. They are also inferior to being able to undo changes.
That's a lesson many don't know.
1 reply →
I can definitely confirm our initial reaction was "WTF" followed with idea that the dev team is making fun of us... but we went in and run traceroutes and there it was :O
Was fixed in incredible coincidence manner, too - the CTO of the network link provider was in their offices (in the same building as me) and felt bored. Apparently having went through all the levels from hauling cables in datacenter up to CTO level, after short look at traceroutes he just picked a phone, called NOC, and ordered a line card replacement on the router :D