Comment by ankurdhama
3 days ago
So the "hardware failure" happening exactly at the same time the Windows update installation failed are not related? That sounds like a one in a billion kind of coincident.
3 days ago
So the "hardware failure" happening exactly at the same time the Windows update installation failed are not related? That sounds like a one in a billion kind of coincident.
An upgrade process involves heavy CPU use, disk read/writes, and at least a few power cycles in short time period. Depending what OP was doing on it otherwise, it could've been the highest temperature the device had ever seen. It's not so crazy.
My guess would've been SSD failure, which would make sense to seem to appear after lots of writes. In the olden days I used to cross my fingers when rebooting spinning disk servers with very long uptimes because it was known there was a chance they wouldn't come back up even though they were running fine.
Not for a server, but many years ago my brother had his work desktop fail after he let it cold boot for the first time in a very long time.
Normally he would leave his work machine turned on but locked when leaving the office.
Office was having electrical work done and asked that all employees unplug their machines over the weekend just in case of a surge or something.
On the Monday my brother plugged in machine and it wouldn’t turn on. Initially the IT guy remarked that my brother didn’t follow the instructions to unplug it.
He later retracted the comment after it was determined the power supply capacitors had gone bad a while back, but the issue with them was not apparent until they had a chance to cool down.
> In the olden days I used to cross my fingers when rebooting spinning disk servers with very long uptimes because it was known there was a chance they wouldn't come back up even though they were running fine.
HA! Not just me then!
I still have an uneasy feeling in my guts doing reboots, especially on AM5 where the initial memory timing can take 30s or so.
I think most of my "huh, its broken now?" experiences as a youth were probably the actual install getting wonky though, rather than the few rare "it exploded" hardware failures after reboot, though that definitely happened.
This, 100%.
I'd like to add my reasoning for a similar failure of an HP Proliant server I encountered.
Sometimes hardware can fail during long uptime and not become a problem until the next reboot. Consider a piece of hardware with 100 features. During typical use, the hardware may only use 50 of those features. Imagine one of the unused features has failed. This would not cause a catastrophic failure during typical use, but on startup (which rarely occurs) that feature is necessary and the system will not boot without it. If it could, it could still perform it's task... because the damaged feature is not needed. But it can't get past the boot phase, where the feature is required.
Tl;dr the system actually failed months ago and the user didn't notice because the missing feature was not needed again until the next reboot.
Is there a good reason why upgrades need to stress-test the whole system? Can't they go slowly, throttling resource usage to background levels?
They involve heavy CPU use, stress the whole system completely unnecessary, the system easily sees the highest temperature the device had ever seen during these stress tests. If during that strain something fails or gets corrupted, it's a system-level corruption...
Incidentally, Linux kernel upgrades are not better. During DKMS updates the CPU load skyrockets and then a reboot is always sketchy. There's no guarantee that something would not go wrong, a secure boot issue after a kernel upgrade in particular could be a nightmare.
2 replies →
> Depending what OP was doing on it otherwise, it could've been the highest temperature the device had ever seen. It's not so crazy.
Kind of big doubt. This was probably not slamming the hardware.
That was absolutely slamming the hardware. (source: worked on Android, and GPs comments re: this are 100% correct. I’d need a bit more, well anything, to even come around to the idea the opposite is even plausible. Best steelman is naïvete, like “aren’t updates are just a few mvs and a reboot?”)
Over my 35 years of computer use, most hardware failures (very, very rare) happen during a reboot or power-on. And most of my reboots happen when installing updates. It actually seems like a very high probability in my limited experience.
Of course, it’s possible that the windows update was a factor, when combined with other conditions.
There's also the case where the hardware has failed but the system is already up so it just keeps running. It's when you finally go to reboot that everything falls apart in a visible manner.
This is one of the reasons I am not a fan of uptime worship. It's not a stable system until it's able to cold boot.
Say you have a system that has been online for 5 years continuously until a power outage knocks it out. When power is restored, the system doesn't boot to a working system. How far back do you have to go to in your backups to find a known good system? And this isn't just about hardware failure, it's an issue of configuration changes, too.
I also notice that people with lots of experience with computers will automatically reboot when they encounter minor issues (have you tried turning it off and on again?).
When it then completely falls apart on reboot, they spend several hours trying to fix it and completely forget the "early warning signs" that motivated them to reboot in the first place.
I've think the same applies to updates. I know the time I'm most likely to think about installing updates is when my computer is playing up.
4 replies →
For all we know, this thing was on its last legs (these machines do run very hot!) and the update process might have been the final nail in the coffin. That doesn't mean Microsoft set out to kill OP's machine... Same thing could have happened if OP ran make -j8 -- we wouldn't blame GNU make.
This reminds me of the 3090 hardware problems being exposed by Amazons New World [1]. Everyone really wanted to blame the software.
https://www.pcgamer.com/amazon-new-world-killing-rtx-3090-gp...
I had a friend's dad's computer's HDD fail while I was installing Linux on it to show him it. That was terrifying. I still remember the error, and I just left with it (and Windows) unable to boot. Later my friend told me that the drive was toast.
Come to think of it, maybe it was me. I might have trashed the MBR? I remember the error, though, "Non system disk or disk error".
IIRC, that error text comes from the mbr. You may have trashed the partition table?
Yeah, I think so. It's been ~25 years, and only while typing out that comment did I remember the error message and realize that's probably what I had done.
If I recall correctly, he ended up scrapping the drive.
Yeah, sounds like the drive was still physically detected but that the expected boot loader wasn't present any more.
If had happened any other time, there wouldn't be a blog post about it and we wouldn't be reading about it.
I've fixed thousands of PCs and Macs over my career. Coincidences like this happen all the time. I mean, have you seen the frequency of updates these days? There are always some kind of updates happening. So the chances of your system breaking during an update is not actually that slim.
[dead]
I think it's fair to say they're related, yes. But causality can well be the other way around — that Windows upgrade failed because of flaky hardware.
Two bugs occurring at the same time is definitely not one in a billion, and with billions of computers in the world, weird shit is going to happen.
> That sounds like a one in a billion kind of coincident
Hardware is more likely to fail under load than at idle.
Blaming the last thing that was happening before hardware failed isn't a good conclusion, especially when the failure mode manifests as random startup failures instead of a predictable stop at some software stage.
A software update can absolutely trigger or unmask a hardware bug. It’s not an either/or thing, it’s usually (if a hardware issue is actually present) both in tandem:
windows update just doing a normal write causing the active chunk of flash memory being used to hold something in the boot loader to a different failed/failing section
This happens all the time, people always doubt it - but the patterns are always consistent: large updates kill hardware that's in progress of failing
"Hardware failure" => "WinUpdate failure" => "Corrupted system" conforms the Occam's razor.
Like winning the lottery?
Happens quite often