Comment by krisoft

Oh I have a similar one. This one is first hand, I was in the room when we were debuging the issue.

We were developing a smart camera product which were counting traffic on a road. So for example a city council would install this camera somewhere on a road and it would generate statistics of how many lorries, and passenger vehicles, and motorbikes used that road.

One of our cameras exhibited a problem where it restarted every day around roughly the same time. It wasn't exactly the same second though, in fact there was a clear pattern to it. One day it would restart at 19:12:10 and the next day two second later, then again the third day two more seconds later. (not the real timestamp and i don't remember the real time deltas either, but there was a clear progression)

After much debuging we learned that the issue was that as the sun was settling some street furniture projected a shadow in front of our camera. Our software wrongly concluded that it is a vehicle and started collecting information about it for classification. But of course shadows creep a lot slower than real vehicles so it run out of memory before the "shaddow vehicle" has passed out of the frame. And once we run out of memory the system froze and then got restarted by a watchdog.

Turns out the pattern we have seen in the timestamps was caused by the angle of the sun changing which made the shadow trick our algorithm just a little bit later every day.