Comment by ack_complete

I did extensive MMX and SSE2 optimization of audio and video codecs in the 2000s. MMX made a large difference, but it was a pain.

MMX optimization practically required assembly language. The Pentium MMX was an in-order dual pipe CPU, and while compilers supported MMX intrinsics, their code generation for it was abysmal. Visual C++ 6, for instance, would emit code that was 2/3rds register-to-register moves, with values being unnecessarily moved between two registers between each ALU op. This was also a problem with SSE/SSE2 intrinsics. The worst case I saw was the _mm_set_epi8() intrinsic, which was used to construct a 128-bit vector from 16 inputs. When used with all constants, it should have generated a single 128-bit constant load; instead, Visual Studio 2008 generated ~80 instructions to compute it from byte loads. Microsoft didn't fix it until VS2010.

The latency of MMX instructions combined with the in-order dual pipe architecture also made asm loops messy. Simple ops were single-cycle, but multiplies had 3 cycle latency, stores required data an additional cycle in advance, and computed load/store addresses were also needed a cycle in advance. Simply running 2-4 iterations in parallel wasn't an option as there were only 8 vector registers and you'd still get bottlenecks on functional units. Getting peak performance thus often required interleaving loop iterations with special entry and exit code around the loop.

The issue with EMMS is understated. When the CPU switched to MMX, it marked the entire x87 stack as full. If you forgot the EMMS instruction, it wasn't just some strange floating-point bugs that would happen -- the next few x87 floating point calculations could just outright produce NaNs due to FP stack overflow. Furthermore, as these NaNs propagated, the CPU required microcode assists to handle them. So, even if the program didn't crash, an entire calculation domain would get poisoned and slow down by ~20x.

Ultimately, I don't think SSE2 was what killed MMX, but rather SSE, and specifically floating point. MMX not only didn't support floating point, but was also highly concentrated on 16-bit signed integers and secondarily 8-bit unsigned integers. Support for 32-bit integers was particularly lacking and pack/unpack conversions were a bottleneck. Trying to do 3D was cramped because doing so required fixed-point and MMX didn't have the same affordances as DSPs or NEON for rounding or implicit narrow/widen in operations, or even swizzles. SSE, on the other hand, was just straight floating point with standard automatic IEEE rounding and also had important added operations like swizzles and insert/extract. Thus, when 3D took off, SSE was far more useful than MMX.

MMX, however, still remained useful for a while for image and signal processing. SSE2 being twice as wide didn't help algorithms that couldn't use the greater width, such as 8x8 block motion prediction. Additionally, some CPUs at the time only had a 64-bit data path and had to split SSE2 ops, but because of their 4-1-1 decode template, could only decode one such instruction per cycle. The result was that code using the MMX registers could still run noticeably faster than with the SSE registers. This caused some confusion with the 64-bit version of Windows since Microsoft tried to say that x87/MMX shouldn't be used in long mode, but after queries from video processing companies had to document that the x87/MMX registers were enabled and context switched for user mode code.