Comment by jordigh
2 years ago
Let me see if I'm the first one to link to that classic story in the same series, "I cannot send email further than 500 miles"
http://www.ibiblio.org/harris/500milemail.html
Or the Magic/More Magic switch
http://www.catb.org/jargon/html/magic-story.html
It's fun when physical reality meets the abstract models that we have built in our heads of these machines.
Oh man, this is one of my favorite lines of all time:
> "Anyway, I asked one of the geostatisticians to look into it--"
> "Geostatisticians..."
> "--yes, and she's produced a map showing the radius within which we can send email to be slightly more than 500 miles. There are a number of destinations within that radius that we can't reach, either, or reach sporadically, but we can never email farther than this radius."
I adore when experts use their expertise to analyze real-world things like this and provide ridiculously thorough explanations :-D
My favorite story kinda of this nature, of an expert as alien intelligence, was Feynmann's calculations about computer architecture of the Connection Machine:
https://longnow.org/essays/richard-feynman-connection-machin...
It's a few paragraphs, maybe too much to quote, but the bulk of it starts with:
> By the end of that summer of 1983, Richard had completed his analysis of the behavior of the router, and much to our surprise and amusement, he presented his answer in the form of a set of partial differential equations. To a physicist this may seem natural, but to a computer designer, treating a set of boolean circuits as a continuous, differentiable system is a bit strange. [...] Our discrete analysis said we needed seven buffers per chip; Feynman's equations suggested that we only needed five. We decided to play it safe and ignore Feynman.
Guess who was right.
The whole essay is worth reading, if you haven't yet.
That was a great read. It was also interesting to read about what happened to the company.
https://en.m.wikipedia.org/wiki/Thinking_Machines_Corporatio...
One thought I had while reading this was what areas of technologies are still open to amateurs.
I hadn't seen that before, so thanks for posting it. What a great story!
it's a useful analysis. Nobody thought of router hops, but this pattern is pretty much what you'd expect, so it was a very good hint.
My recent version: I was playing a pinball game in an arcade. One particular ramp shot was registering earlier in the day and then stopped working.
Eventually I realized that the sensor is an optical beam, and the receiver happened to be in direct sunlight coming in through a window! So it was continuously receiving infrared and would never report the beam being blocked by a pinball. Sure enough, it started working again once the sun angle changed by a few more degrees.
I have an optical smoke detector that will give (very loud) false alarms if a sun beam can bounce off a windowsill onto it. It works great if the curtain is closed. Debugging that took a few early sunrises.
You missed an opportunity to cheat the machine by waving your hand between it and the sun. ;)
Heh, but not exactly. If I blocked the sun, the receiver still would have been picking up the real beam. I would have had to block the sun _and_ make the shot with a pinball at the same instant... which is just playing the game normally with extra steps.
Or the "Car allergic to vanilla ice cream" story [1].
[1] https://news.ycombinator.com/item?id=37584399
The 500 mile email story is one of my favorite reminders that, fundamentally, we're still governed by the laws of physics. It's funny, but it's also a reminder that, while networks might be very fast, the latency is still going to be governed by the speed of light.
If we are doing classic stories - Grace Hooper and the Nanosecond of wire
https://www.youtube.com/watch?v=9eyFDBPk4Yw
https://americanhistory.si.edu/collections/nmah_692464
Well laws of physics is what gave us radio in the first place.
Some of my favorite video documentaries are on how it was theorized and then slowly developed over years and decades until they finally got to spark-gap transmitters.
https://en.wikipedia.org/wiki/Timeline_of_radio
But just imagine listening to spark-gap morse code radio broadcasts for years as amateur and then suddenly someone does a broadcast test of actual voice (violin!) That must have been incredible to hear wirelessly.
24 December 1906 Reginald Fessenden, that was the leap that eventually gave us wifi
https://en.wikipedia.org/wiki/Reginald_Fessenden
I actually like thinking about the exchange of physical information as a network propagation delay, and entanglement/coherence as a distributed consensus algorithm. They're kinda samey from a conceptual point of view (in my amateur opinion)
I forgot all about the 500 miles story. My favorite line:
> If the problem had had to do with the geography of the human recipient and not his mail server, I think I would have broken down in tears.
I had a customer who used a line of sight system for extending their network across part of a city.
I had a shortcut on my desktop with the weather for that town ready when they would inevitably call and blame our unrelated equipment for some problem.
I worked at a small, local ISP in the 90:ies that had a point to point link across the river, handling the dial up traffic from the telecom company we partnered with.
Every few days, always at roughly the same time, all incoming dial up traffic would drop. A minute later, the customers could reconnect.
It took a while before we realized that one of the huge passenger ferries that docked a short distance upstream was the cause. When it arrived and departed, its chimneys and possibly bridge and highest deck blocked LOS across the river.
I used to work in high-frequency trading. I had several tabs permanently open to the live weather radar feed for regions where we had microwave towers: the NE USA, the South of England, the Alps...
I used to be in an adjacent field and we used to joke about when the HFT guys were gonna get working on some neutrino detectors & sources to signal straight through the Earth. You could use them for science on the weekends!
3 replies →
I'm curious to know where your towers were. Do you know if they still exist? Were your microwave antennae co-located on other operators' towers (e.g. those for VHF radio), or did your company have towers all to itself?
3 replies →
There was a site with stories like these somewhere, I sadly can't remember the URL any more.
I think the one that stuck out to me was the Soviet mainframe computer that would get weird bit flips almost every day, always at the exact same time. Somebody compared what was different about the days it didn't get bit flips on, it turns out those were the days on which a particular train didn't run, the computer was very close to a railway station. What train was it, you ask? The one transporting the (definitely perfectly safe to eat, definitely not filled to the brim with nuclear radiation) cow meat from Chernobyl. The radiation was intense enough to cause bit flips, I'm sure the quality of soviet components didn't help here either.
I think is the one you are looking for:
https://beza1e1.tuxen.de/lore/crash_cows.html
Perhaps thedailywtf.com?
What was the one for Apple stories...?
1 reply →
As someone with very limited electrical experience, the more magic switch story instantly went "the second terminal of the switch is probably grounded to the switch casing" when they explained it only had one connected terminal.
This is a very common thing in older automotive electronics, for example.
First thing I thought of too. Anyone know if there a list of more articles similar to these three?
My wife has complained that open office will never print on Tuesdays!?
https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...
A user was having a really bizarre problem: They could log in when they were sitting down in a seat in front of the keyboard, but when they were standing in front of the keyboard, their password didn't work! The problem happened every time, so they called for support, who finally figured it out after watching them demonstrate the problem many times:
It turned out that some joker had rearranged the numbers keys on the keyboard, so they were ordered "0123456789" instead of "1234567890". And the user's password had a digit in it. When the user was sitting down comfortably in front of the keyboard, they looked at the screen while they touch-typed their password, and were able to log in. But when they were standing in front of the computer, they looked at the keyboard and pressed the numbers they saw, which were wrong!
I have one first-hand story:
I did tech support via phone for a popular consumer computer brand. One particular call, a woman reported that her computer was restarting every time someone in the house flushed the toilet.
Long story short, her home was in the back-back woods with the home powered by a generator. In addition to powering the computer, the generator was also the source of power for a water pump which would kick on to refill the toilet bowl whenever it emptied. And wouldn't you know that that water pump had a beefy coil around its motor and would brownout the entire house every time it started?
I have a similar one, with an automated monorail hoist. The engineer who started the job had ordered the monorail hoist with a control cabinet with Ethernet comms to tell it where to move (instead of just controlling the hoist directly from the main control cabinet.) After days' worth of shenanigans trying to troubleshoot seemingly random comms drop-outs I'd narrowed it down to only occurring when the hoist was being lowered under load, which led me to the Ethernet cable in the hoist cabinet which ran parallel to the motor cables from the hoist's 6kW VSD. Whenever it lowered, the EMI was enough to nuke the Ethernet connection. Re-routed the Ethernet cable and after that it ran fine.
My personal example: VoIP phones stopping after the Asterisk server was up for 3 days.
Reason: the server had IPv6 turned on, and it steadily accumulated privacy IPv6 addresses. These addresses were all sent in a packet describing the supported media endpoints, using UDP.
And yep, eventually it overflowed the MTU and the phones couldn't handle the fragment reassembly.
Which distro was that? ... asking for a friend ...
4 replies →
Here's one attempt I've seen in other HN comments at a shared "awesome list" of these sorts of stories:
https://github.com/danluu/debugging-stories
The podcast that kills the car stereo episode of Reply All is pretty funny https://gimletmedia.com/amp/shows/reply-all/brh8jm
I remember one (might have been a hn-er's comment, dunno) about the computer restarting when the toilet was flushed. Turns out it was due to voltage drop when a compressor turned on to refill the reservoir of the toilet.
That's why in rural locales with spotty power it pays to have a UPS on any electronics -- you might not benefit much from 15-30 minutes of extra power in a day long blackout, but it keeps everything happy when the voltage fluctuates.
It was:
https://news.ycombinator.com/item?id=39898272
https://500mile.email although I do wish it had more content!
Thanks for mentioning! I would love more submissions! I have a few stories in my backlog to read and vet, but not enough. I'll be going through this thread and adding more that haven't been added yet.
The Daily WTF is full of them.
https://thedailywtf.com
DNS responses sent over UDP are often truncated if the response is too large. This manifests itself as "machine unreachable if name > x characters" sort of errors when you have really long FQDNs.
There's the car/ice-cream/vapor-lock story. Oops, I ruined it.
https://www.snopes.com/fact-check/cone-of-silence/
This is a true classic that never gets old!
Discussed many times here in HN:
1. https://news.ycombinator.com/item?id=9338708
Magic/more magic legend lives! I tell that to everyone experiencing a spooky troubleshooting!
i love those stories plus their detailed explanations because you can learn so much from them about technology, physics and even psychology.
and previously discussed: https://news.ycombinator.com/item?id=9338708
I came here to see if someone posted that.
Boy did that [0] send me down a long rabbit hole
[0] magic-story