Comment by exmadscientist

> in this particular case I think it's a little misguided ... only inherent difference between 8-bit and 32-bit code is the use of `movzx` versus `mov` ... generates ugly code for address calculation ... the induction heuristics didn't do their best here

OK, it seems I've been a bit confused myself. I think my final conclusion is actually just that clang is generating kind of crap code here, and this change gets good codegen out of clang. Since I was mostly looking at clang and was leaving gcc as an afterthought, I didn't pick up on this. (Obviously I'm a little rusty on my x86, I shouldn't have fallen for that!) As gcc shows, it doesn't matter much either way. But when clang is happy, its code is quite nice, so there's that.

That little address dependency chain is nastier than it looks, though, the way clang is writing it. Best done away with. There's really no reason for clang to be emitting that crud. gcc handles the addressing in a much cleaner manner whether it's byte- or dword-wide. But gcc also does need that fence when clang does not. I'm sure that's not a coincidence.

> Could you explain your thought process [on placing the compiler fence]?

I ended up dumping this in kind of carelessly and noting that it worked before moving on (it was a last-minute add and I was kind of done by this point). But GCC does care where it goes, since it doesn't work if placed before. Let's try this explanation: the fence basically means "resolve all funny business with this variable before proceeding". Before the write, there's no funny business to resolve: nothing has happened to that variable. No writes, no outstanding reads (we're well into the loop at this point, in theory). So it's a fancy nop. But if it's after the write, we've now got a consequence: `next_j[i][j]` is different now. The fence forces us to resolve that now and not let it linger until the next loop iteration. Where it can be handled perfectly well, but not particularly quickly (which we do care about), and by the way that choice puts the update in the fast path. Forcing it to be dealt with before we re-enter the fast path is how we force the stupid compiler to keep it out of the fast path.

> ugly code for address calculation... might affect throughput a little

What I'd actually worry most about is poisoning the prefetchers, and speculation more generally. They're very very good at handling things like "increasing stride off this base". They struggle more with things like "double indirect register access to determine next address", though there is so much compiler crud out there that they have some surprising capabilities. Basically: simple stupid loop stride is going to do well with them. Complicated four-instruction chains to generate an address that could have been `[base + 8*stride]` or whatever is just... not helping. gcc does perfectly well here. It has multiple offsets floating around, but all of those instructions to update them can update superscalar, in a single cycle. The clang chain is a four-cycle delay. And to top it all off the branch can probably fuse in the simple addressing case!