Comment by 1970-01-01
11 hours ago
>Across multiple runs of each test, the Snapdragon system produced consistent, repeatable timings nearly every time. On the Intel system, results varied significantly, occasionally beating the Snapdragon, but most of the time falling behind. The Snapdragon was the clear winner on each test overall.
They blogged everything to generate the setup, including the hunch and test code but the anecdotal results are missing. It's a little suspect. How much faster is ARM??
I intentionally left out screenshots of the output for a couple of reasons:
1) They’d distract from the main point (I wasn’t aiming to write a benchmarking post), and
2) They can be misleading, since results will vary across ARM hardware and even between Snapdragon X Elite variants.
Instead, I included the PowerShell snippets so anyone interested can reproduce the results themselves.
For a rough sense of the outcome: the Snapdragon VM outperformed the Intel VM by ~20–80%, depending on the test (DNS ~20%, IIS ~50%, all others closer to ~80%).
> so anyone interested can reproduce the results themselves.
Weird reasoning.
You already caught our attention with your article. But not everyone has the time or means to go and re-do the tests.
However such information is really important to surface when making infra decisions. And if one of the brain cells pops up and says something about 20-80% perf improvement VS there were some perf improvements - which would be more convincing to research the topic when the time comes for the reader to benefit from your research?
You likely tripped over a difference in power management profiles (and capabilities) between Intel and ARM.
You're testing "variability" and latency, and you even mention that "modern Intel CPUs tend to ramp frequency..." but entirely neglect to mention which specific Windows Power Profile you were using.
Fundamentally, you're benchmarking a server operating system on laptops and/or desktop-class hardware, and not the same spec either. I.e.: you're not controlling for differences in memory bandwidth, SSD performance, etc...
Even on server hardware the power profiles matter! A lot more than you think!
One of my gimmicks in my consulting gig is to change Intel server power settings from "Balanced" to "Maximum Performance" and gloat as the customer makes the Shocked Pikachu face because their $$$ "enterprise grade server" instantly triples in performance for the cost of a button press.
Not to mention that by testing this in VMs, you're benchmarking three layers: The outer OS (and its power management), the hypervisor stack, and the inner guest OS.
Both Windows 11 systems are configured with the “High performance” power plan, as are the two Windows Server VMs. In hindsight, I should have included this detail explicitly in the original post instead of only alluding to it.
6 replies →