Comment by adrian_b

When designing CPU cores with SIMD execution units, it is very important to ensure that data can be loaded from and stored to the L1 cache memory at a throughput high enough to not starve the SIMD execution units.

Thus normally any CPU core that has a good SIMD throughput will also have a good throughput for transfers between the vector registers and the L1 cache memory.

The CPU cores of the last 3 decades that are considered well balanced can do the same number of FMA instructions per clock cycle (fused multiply-add) and vector register loads from the L1 cache memory. The throughput for stores is usually a half of that for loads, as that is typically enough.

For instance an AMD Zen 5 core in full configuration can do in every clock cycle two 512-bit FMA + two 512-bit loads + one 512-bit store (as a bonus, it can also do two 512-bit additions). A similar throughput is provided by the Intel P-cores used in their server CPUs.

If a CPU core would not have a memory load throughput proportional with the SIMD execution throughput, then much fewer programs would be able to reach a speed limited by the arithmetic operations, and most would be limited by the memory load throughput.

Even with the compromise currently chosen for designing CPU cores, the programs that use SIMD instructions are clearly divided into 2 classes, one where the performance is limited by the throughput of executing arithmetic operations and one where the performance is limited by the throughput of the load/store operations.