Comment by mancerayder
1 day ago
The arstechnica article was very good as a history of waterfall v sprint using MS as a case study. However the firing the QA department narrative is not supported:
Prior to these cuts, Testing/QA staff was in some parts of the company outnumbering developers by about two to one. Afterward, the ratio was closer to one to one. As a precursor to these layoffs and the shifting roles of development and testing, the OSG renamed its test team to “Quality.”
Two QA per dev?? That seems ginormous to me. What am I missing about the narrative about evil corp sending all of QA packing, that seems not supported here?
The second, Reuters article seems like it's saying something different than the QA firing narrative - it seems to talk about Nokia acquisition specifically and a smattering of layoffs.
Not supporting layoffs or eliminating QA, and I'm deeply annoyed at Windows 11. I just don't see these as supportive of the narrative here that QA is kaput.
> Two QA per dev?? That seems ginormous to me. What am I missing about the narrative about evil corp sending all of QA packing, that seems not supported here?
I think you're underestimating the QA burden for large parts of the company. When I worked in payments at MS, the ratio of QA to dev after the cuts was probably on the order of dozens to one, if not a hundred or more once you threw in Xbox/Windows/etc accessibility QA from across the organization and all the other people like lawyers involved in handling over a hundred jurisdictions. I was little more than a frontend line cook and even I had three QA people reporting directly to me; two of them helping write tests so they ostensibly should have been automating themselves out of a job.
There is a lot of manual testing when you have a complex system like that where not everything can be properly stubbed out, emulated, or replaced with a test API key. They also have to be kept around to help with painful bursty periods (for us it was supporting PSD2, SCA, or 3DS2, forgot which). Payments is obviously an outlier because there is a lot of legal compliance, but the people I knew in Cloud/Windows also had lots of QA per dev.
I wouldn't be surprised if the degradation in feature parity of newer Windows software was a result of this loss of QA. Without the QA, the developers have to be less ambitious in what they implement in order to meet release schedules, and since they don't have experienced QA they can't modify the older codebases at all to extend them.
Remember also, they were doing an enormous amount of testing with third-party devices and software*. Which is what seems to keep blowing up most spectacularly. Even if something works on 99.9% of computers, with a billion installs that's a few million dissatisfied customers
* Stories like this: https://devblogs.microsoft.com/oldnewthing/20250610-00/?p=11...
In writing life critical systems like the Space Shuttle's operating system, effectively 99.9% of all work is QA.
MS had the dominant operating system in the world, and keeping its userbase and its ~monopoly dividend would have been more profitable as a business than doing... everything it's done in the past twenty years. Selling software that all the people use all the time just has a lot less opportunity for growth than making new software, according to Investor Brain.
>In writing life critical systems like the Space Shuttle's operating system, effectively 99.9% of all work is QA.
Similar in automotive safety related systems like brake, steering or powertrain.
Few is writing function code to how much is requirement engineering, FMEA and writing tests and testing.
The Windows ecosystem is insanely complex. And they supported it, because of the focus on QA and testing the company adopted 20 years ago after the Blaster worm.
I have a few pretty awesome teams stuck managing windows. They find bugs all of the time. The process of fixing them now practically requires a detachment of druids and Stonehenge to track where in the windows/lunar/solar cycles we are and how to deal with the bullshit & roadblocks the support and product teams throw up. If you fall for their tricks, you’ll miss the feature window… no fix for 18 months.
It used to be much easier as a customer in ye olden times, and I never felt that the counterparty at Microsoft was miserable or getting punished for doing their jobs. We feel that now as customers. You didn’t establish relationships with engineers like with other vendors, but there was a different vibe.
The focus of the company moved in to Azure, service ops, etc.
I worked in the windows org around that time and the Dev/QA ratio there was closer to 1:1. QA did both manual testing and much of the automation, quality gates, and did regression testing against older versions of windows. Given the complexity of the product is is fairly easy for an inexpensive change to require an expensive test effort.
I had a QA engineer who gave me feedback on designs, great code reviews, and who wrote tests that I could also run.
It was a partnership. I miss it.
And honestly, that person deserves the same pay grade as a "normal" engineer. But sadly, most QA staff are underpaid and somewhat even an inferior class.
Instead, if the QA role was the dominant and better paid title, you'd immediately see an improvement in that partnership. I don't think that you need subordinate staff in the QA role at all.
And for what its worth, I'm that guy. I am a strong technical software developer, but I would much rather test and poke at code bases, finding problems, working with a "lead" developer, and showing them all their quality mistakes. If I could have that role at my pay grade, I'd be there.
Quality testers are so extremely valuable.
In the chip design world, 2:1 for design verification to design is on the low end of normal.
Some organizations have gone as low as 1:1 but that is considered an emergency that must be fixed. It’s so important that designers will be intentionally underworked if there are not enough validation engineers on staff.
When you can’t fix bugs in the field, quality is important.
> Two QA per dev??
QA is a lot cheaper than dev. If your goal is to make quality software* on a fixed budget, you want to be QA-heavy.
* Note: the OS definition of "quality software" drastically differs from your average app.
> QA is a lot cheaper than dev.
QA is definitely one of those "you get what you pay for". A dev just bangs out code on what is assumed "happy path" which means the user uses it as the dev expects. QA has to some how think of all the inane ways that a user will actually try using the thing knowing that not all users are technically savvy at all. They are actively trying to break things not just feed in clean data to produce expected outputs. Let's face it, that's exactly what devs do when they "test". They are specifically trying to get unexpected outputs to see how things behave. At least, good QA teams do.
I worked with a QA person who I actively told anyone that listened that the specific QA person deserved a higher salary than I did as the dev. They caught some crazy situations where product was much better after fixing.
> QA has to some how think of all the inane ways that a user will actually try using the thing knowing that not all users are technically savvy at all.
The classical joke is: (this variant from Brenan Keller[0])
A QA engineer walks into a bar.
- Orders a beer.
- Orders 0 beers.
- Orders 99999999999 beers.
- Orders a lizard.
- Orders -1 beers.
- Orders a ueicbksjdhd.
First real customer walks in and asks where the bathroom is.
The bar bursts into flames, killing everyone.
[0] https://xcancel.com/brenankeller/status/1068615953989087232?...
That's just a bad dev. Good devs don't think of just the happy path. My experience of QA as a quality focused dev has not been good.
1 reply →
I feel that not only should QA staff outnumber developers, but QA staff should have access to development time to design and improve QA tooling.
If you're doing an OS right, the quality is the product. I think MacOS prior to the launch of the iPhone would be the gold standard the kind of product design I'm talking about. At that time they were running circles around Windows XP/7 in terms of new features. They were actually selling the new OSes and folks were happy to pay for each roughly annual upgrade. Often the same hardware got faster with the newer OS.
Lately Microsoft and Apple are racing to the bottom, it seems.
The irony here is that the market is willing to pay for quality.
I don't have time to deal with phone issues-it should just work so I can get on with my day.
Hearing that Apple were dedicating time to stop features and go after stability is exactly what I want to hear.
2 replies →
Important to note MS used to have 2 types of QA:
1. SDETs (software design engineer in test) - same pay scale and hiring requirements as SDEs, they did mostly automated testing and wrote automated test harnesses.
2. STEs (software test engineer) - lower pay scale, manual testing, often vendors. MS used to have lots of STE ftes but they fired most of them in the early 2000s (before I joined in 2007).
An ideal ratio of SDETs to SDEs was 1 to 1, but then SDET teams would have STE vendors doing grunt work.
Having STEs as full time employees benefited MS greatly. They knew products from the end user and UI/UX perspective inside and out in ways even the SDETs didn't.
UI/UX quality in MS products dipped noticeably after the STE role was eliminated.
1 reply →
2 people doing QA per dev seems insane even if it’s a lot cheaper. M$ is hardly know for being obsessed with quality, they’d rather have 2 sales per dev (sales is even cheaper, basically pays for itself)
It's a lot easier to write code than to make sure it doesn't break something you didn't account for.
Microsoft's quality control used to be: release a buggy new OS then forge a solid next release.
Going backwards in stability is out of character.
I've never known M$ to be lacking on the sales front, personally!
> Two QA per dev?? That seems ginormous to me.
The only person I heard was writing perfect code was Donald Knuth. And even he had bugs in its code.