A user was having a really bizarre problem: They could log in when they were sitting down in a seat in front of the keyboard, but when they were standing in front of the keyboard, their password didn't work! The problem happened every time, so they called for support, who finally figured it out after watching them demonstrate the problem many times:
It turned out that some joker had rearranged the numbers keys on the keyboard, so they were ordered "0123456789" instead of "1234567890". And the user's password had a digit in it. When the user was sitting down comfortably in front of the keyboard, they looked at the screen while they touch-typed their password, and were able to log in. But when they were standing in front of the computer, they looked at the keyboard and pressed the numbers they saw, which were wrong!
I did tech support via phone for a popular consumer computer brand. One particular call, a woman reported that her computer was restarting every time someone in the house flushed the toilet.
Long story short, her home was in the back-back woods with the home powered by a generator. In addition to powering the computer, the generator was also the source of power for a water pump which would kick on to refill the toilet bowl whenever it emptied. And wouldn't you know that that water pump had a beefy coil around its motor and would brownout the entire house every time it started?
I have a similar one, with an automated monorail hoist. The engineer who started the job had ordered the monorail hoist with a control cabinet with Ethernet comms to tell it where to move (instead of just controlling the hoist directly from the main control cabinet.) After days' worth of shenanigans trying to troubleshoot seemingly random comms drop-outs I'd narrowed it down to only occurring when the hoist was being lowered under load, which led me to the Ethernet cable in the hoist cabinet which ran parallel to the motor cables from the hoist's 6kW VSD. Whenever it lowered, the EMI was enough to nuke the Ethernet connection. Re-routed the Ethernet cable and after that it ran fine.
My personal example: VoIP phones stopping after the Asterisk server was up for 3 days.
Reason: the server had IPv6 turned on, and it steadily accumulated privacy IPv6 addresses. These addresses were all sent in a packet describing the supported media endpoints, using UDP.
And yep, eventually it overflowed the MTU and the phones couldn't handle the fragment reassembly.
I remember one (might have been a hn-er's comment, dunno) about the computer restarting when the toilet was flushed. Turns out it was due to voltage drop when a compressor turned on to refill the reservoir of the toilet.
That's why in rural locales with spotty power it pays to have a UPS on any electronics -- you might not benefit much from 15-30 minutes of extra power in a day long blackout, but it keeps everything happy when the voltage fluctuates.
Thanks for mentioning! I would love more submissions! I have a few stories in my backlog to read and vet, but not enough. I'll be going through this thread and adding more that haven't been added yet.
DNS responses sent over UDP are often truncated if the response is too large. This manifests itself as "machine unreachable if name > x characters" sort of errors when you have really long FQDNs.
My wife has complained that open office will never print on Tuesdays!?
https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...
A user was having a really bizarre problem: They could log in when they were sitting down in a seat in front of the keyboard, but when they were standing in front of the keyboard, their password didn't work! The problem happened every time, so they called for support, who finally figured it out after watching them demonstrate the problem many times:
It turned out that some joker had rearranged the numbers keys on the keyboard, so they were ordered "0123456789" instead of "1234567890". And the user's password had a digit in it. When the user was sitting down comfortably in front of the keyboard, they looked at the screen while they touch-typed their password, and were able to log in. But when they were standing in front of the computer, they looked at the keyboard and pressed the numbers they saw, which were wrong!
I have one first-hand story:
I did tech support via phone for a popular consumer computer brand. One particular call, a woman reported that her computer was restarting every time someone in the house flushed the toilet.
Long story short, her home was in the back-back woods with the home powered by a generator. In addition to powering the computer, the generator was also the source of power for a water pump which would kick on to refill the toilet bowl whenever it emptied. And wouldn't you know that that water pump had a beefy coil around its motor and would brownout the entire house every time it started?
I have a similar one, with an automated monorail hoist. The engineer who started the job had ordered the monorail hoist with a control cabinet with Ethernet comms to tell it where to move (instead of just controlling the hoist directly from the main control cabinet.) After days' worth of shenanigans trying to troubleshoot seemingly random comms drop-outs I'd narrowed it down to only occurring when the hoist was being lowered under load, which led me to the Ethernet cable in the hoist cabinet which ran parallel to the motor cables from the hoist's 6kW VSD. Whenever it lowered, the EMI was enough to nuke the Ethernet connection. Re-routed the Ethernet cable and after that it ran fine.
My personal example: VoIP phones stopping after the Asterisk server was up for 3 days.
Reason: the server had IPv6 turned on, and it steadily accumulated privacy IPv6 addresses. These addresses were all sent in a packet describing the supported media endpoints, using UDP.
And yep, eventually it overflowed the MTU and the phones couldn't handle the fragment reassembly.
Which distro was that? ... asking for a friend ...
Just regular Ubuntu. It was around 2009 or so.
3 replies →
Here's one attempt I've seen in other HN comments at a shared "awesome list" of these sorts of stories:
https://github.com/danluu/debugging-stories
The podcast that kills the car stereo episode of Reply All is pretty funny https://gimletmedia.com/amp/shows/reply-all/brh8jm
I remember one (might have been a hn-er's comment, dunno) about the computer restarting when the toilet was flushed. Turns out it was due to voltage drop when a compressor turned on to refill the reservoir of the toilet.
That's why in rural locales with spotty power it pays to have a UPS on any electronics -- you might not benefit much from 15-30 minutes of extra power in a day long blackout, but it keeps everything happy when the voltage fluctuates.
It was:
https://news.ycombinator.com/item?id=39898272
https://500mile.email although I do wish it had more content!
Thanks for mentioning! I would love more submissions! I have a few stories in my backlog to read and vet, but not enough. I'll be going through this thread and adding more that haven't been added yet.
The Daily WTF is full of them.
https://thedailywtf.com
DNS responses sent over UDP are often truncated if the response is too large. This manifests itself as "machine unreachable if name > x characters" sort of errors when you have really long FQDNs.
There's the car/ice-cream/vapor-lock story. Oops, I ruined it.
https://www.snopes.com/fact-check/cone-of-silence/