Comment by mlegendre
2 years ago
Your mouse story makes me think of the day the CI system at work turned out not to be robust to vibrations.
One day we started having flaky tests, seemingly out of nowhere. We quickly identified that the issue affected tests involving graphical X client applications, but then we struggled to make further progress. The issue was just impossible to reproduce in other conditions... Well, as it happens, the CI jobs were running on some desktop machines we had installed somewhere within our premises. It turned out that some gentleman had plugged a mouse into one of the machines, and left it lying around on the shelf. Since then, when one of the machines was under a heavy load, the fans would spin faster, causing more vibrations, in turn causing the mouse to move, ever so slightly. And for ungodly reasons, this had side effects on tests.
Fun fact: the machines were not on my site, I managed to diagnose this over SSH. I was quite proud :-)
> And for ungodly reasons, this had side effects on tests.
Let me guess - tests with very tight timings?
Sadly I can't remember this part; I'm pretty sure there were comical bits to it.
This makes me want to dig out the gitlab issue, and turn it into a better write-up! This'll have to wait until I'm back from holidays though.