Comment by DarmokJalad1701

I have encountered at least one bug at $job which was tracked down to x87 instructions. Our "production" build is deployed on embedded ARM CPUs while we have test builds compiled for x86 (32-bit) and x86_64 (different subsets of functionality). Anyway, the bug only showed up in the 32-bit x86 build. The same code worked fine in production and in the 64-bit test builds.

It turned out to be an x87 bug where a piece of code was actually computing the wrong answer!. Logically following the code would make you think that the particular failure in question would never happen - and yet it did. That was quite a rabbit-hole to go into to figure out.