Comment by adrian_b
4 days ago
While it would be nice to have a multi-threaded benchmark that demonstrates a realistic scaling towards many threads, I have no reason whatsoever to trust that the Geekbench implementers are capable to optimize the inter-thread communication well enough for reaching the limits imposed by the hardware.
If they use inefficient thread synchronization methods, like those offered by the C++ compiler, or by the pthreads library on Linux, or by some of the older Win32 functions, their results will be much too pessimistic for the CPUs with many cores, creating the wrong impression that they are not much better, or not better at all, than CPUs with less cores.
For Geekbench 6, I have seen enough results where CPUs that are much better for real multi-threaded workloads appeared to be almost the same as clearly inferior CPUs.
For multi-threaded benchmarks, if they are embarassingly parallel the quality of the implementation does not influence the results, but otherwise it is not possible to trust the results without having access not only to the source program of the benchmark, but also to the compilers used by it.
I have seen a lot of benchmarks of locking methods used for thread synchronization, where the results could not be interpreted correctly without seeing the assembly language instructions generated by the compiler for the locking sequences, because a bad assembly generation could matter more than the high-level algorithms whom the benchmark was supposed to compare.
> While it would be nice to have a multi-threaded benchmark that demonstrates a realistic scaling towards many threads, I have no reason whatsoever to trust that the Geekbench implementers are capable to optimize the inter-thread communication well enough for reaching the limits imposed by the hardware.
You're contradicting yourself. Do you want realistic scaling, or the kind of highly-tuned hand-optimized scaling that almost no real-world applications (especially in the consumer space) ever get?
In both of your comments here, you make very narrow assumptions about what constitutes a "real" workload, and you're excluding ordinary everyday consumer PC software from what you consider "real". Highly optimized software running on large servers aren't the only "real" workloads, and they're not the kind of "real" workload that Geekbench is trying to represent. For the kind of hardware and use cases that Geekbench actually targets, using the platform/compiler default synchronization primitives and doing work in short bursts interspersed with significant idle time is reasonable and realistic.
A very frequent real-world workload is the compilation of a software project, which scales almost perfectly with the number of cores.
For the other multi-threaded tasks, like running certain CAD/EDA applications, I use well-optimized programs, either by others or by myself.
What I do is typical for a professional use of a computer.
The Geekbench multi-threaded results do not provide any useful information about how a computer will behave for such applications.
I agree that the Geekbench multi-threaded results may be representative for how a smartphone will behave if it will have small bursts of multi-threaded activity caused by casually written applications, interspersed between single-threaded execution.
This would be fine, if it would be more clearly specified when the Geekbench benchmark is distributed.
I know that I should trust only the single-threaded Geekbench results and for multi-threaded I should search other published benchmarks.
But others appear to not be aware of this and I have seen frequently erroneous claims on the Internet that some laptop or even some smartphone would match or exceed the performance of a desktop or of a bigger laptop, even one with a CPU with more cores, based on the published Geekbench multi-threaded results, while in reality that desktop or big laptop has a much greater throughput.
> A very frequent real-world workload is the compilation of a software project, which scales almost perfectly with the number of cores.
Geekbench has that as a subtest.
> For the other multi-threaded tasks, like running certain CAD/EDA applications, I use well-optimized programs, either by others or by myself.
Those seem pretty clearly out of scope. You want a workstation benchmark, not a consumer benchmark.
> This would be fine, if it would be more clearly specified when the Geekbench benchmark is distributed.
The descriptions of what Geekbench is are adequate. You're asking for Geekbench to exhaustively list what it isn't to stop you from imagining it to be a one-size-fits-all benchmark, which couldn't possibly exist.
> But others appear to not be aware of this and I have seen frequently erroneous claims on the Internet that some laptop or even some smartphone would match or exceed the performance of a desktop or of a bigger laptop, even one with a CPU with more cores, based on the published Geekbench multi-threaded results, while in reality that desktop or big laptop has a much greater throughput.
Again you seem to be unable to stop asserting that workstation or server use cases are the only ones that are real.