Comment by hayfield

There are a few other things that could also impact results. Looking at the experimental design, there could be a good 25% uncertainty in the results because of how it's done at such a high level.

Rough up-to values for various things I can think of / see:

- 10% because they're using an OS rather than running binaries straight.

- 15% because GCC is odd and -O3 does even stranger things, particularly when it comes to energy.

- 15% because their benchmarks are large workloads rather than microbenchmarks that may better target the architecture rather than being huge lumps (would exacerbate GCC strangeness).

- 17% because they're measuring board rather than CPU power supply (the claim that SoC-based ARM development boards cannot have processor power isolated is questionable - I've seen Beagleboards with the CPU power supply isolated)

- 10% because they're measuring energy consumption at a low resolution (their equipment measures in Hz when there's kit that happily measures in kHz or MHz).

Of course, some of these will cancel, and others will be nowhere near as bad as stated. It also doesn't introduce order-of-magnitude changes to the conclusions, although a few of the 'Key Findings' may want questioning.