Comment by bonzini
1 month ago
Yes, I wish this was this simple. :) There are many other complications:
* Some instructions require VEX.L or VEX.W to be 0 or 1, and some encodings result in completely different instructions if you change VEX.L.
* Different bits of the EVEX prefix are valid depending on the opcode byte.
* Some encodings (called groups) produce different instructions depending on bits 3-5 of the modrm byte (the second byte after all prefixes). Some encodings further produce different groups depending on whether bits 6-7 (mod) of the modrm byte identifies a register or not.
* Some instructions read a whole vector register but only a scalar if the same instruction has a memory operand. Sometimes this is clear in the manual, sometimes it is not, sometimes the manual is downright wrong.
* Some instructions do not allow using the legacy high-8-bits registers even though they don't do anything with bits 8 and above of the operand: they only want a 32- or 64-bit register as their operand.
* APX (EVEX map 4) looks a lot like legacy map 0, but actually a few instructions were moved there from other maps for good reasons, a few more were moved there for no apparent reason (SHLD/SHRD iirc), and a few more are new.
* REX2 does not extend SSE and AVX instructions to 32 registers even though REX does extend them to 16.
* Intel defines a thing called VEX instruction classes, which makes sense except for a dozen or two instructions where it doesn't. For these, sometimes AMD uses a different class, sometimes doesn't; sometimes AMD's choice makes sense, sometimes it doesn't.
And many more that I found out while writing QEMU's current x86 decoder (which tries to be table based but sometimes that's just impossible).
> Some instructions require VEX.L or VEX.W to be 0 or 1, and some encodings result in completely different instructions if you change VEX.L.
There is even an instruction where AMD got this wrong! VPERMQ requires VEX.W=1, but some AMD CPUs also happily execute it when VEX.W=0 even though that is supposed to raise an exception.