← Back to context

Comment by hn_throwaway_99

7 years ago

Wow, this article is pretty amazing to me. I have a joke (err, kind of a joke) that I like to say, "It's never a cosmic ray." What I mean by that is that when you are debugging and getting incredibly frustrated, right when you feel that you want to give up because you can't find the bug, that you shouldn't blame "cosmic rays" because 99.999% of the time it's a logic problem.

Here was a case where it actually was cosmic rays (at least, not a logic problem), and these guys found it. I also agree with the title and intro of the article - I think it's very possible that if these 2 exact people hadn't been at Google at the time than Google's path could have been very different. Makes me also appreciate how much luck is involved in any successful indeavor. Not luck that they found the bug (that was hard work), but luck that these 2 guys found themselves working at Google right when Google needed them.

It’s never a cosmic ray but it’s often bad ram. You don’t have to inspect the binary to diagnose bad ram. The system would be behaving erratically in other ways due to regular memory corruption. It’s an amazing story but by no means the only way it could have played out.