Comment by jcranmer

An optimizer is trying to balance between compile time, runtime speed, and code size, and most optimizations will win you one axis at the cost of one or both of the other axes. (The rare optimizations that win on all 3 are already all implemented in the compiler.) Compiler developer time is also a scarce resource; I know of so many more optimizations I could implement, but without demonstrable code that would actually benefit, it's not a good use of my time to implement them. Compilers tune this balance by making lots of heuristic decisions, and these heuristics are tuned by large benchmarks, which often times involve a lot of flat code profiles (i.e., no code is worth spending a lot of time really nailing down the best code layout).

One of the advantages of hand-written assembly is that you get to opt out of the compiler heuristics and commit to being able to spend the time to optimize the one bit of code that you know is really important for runtime as perfectly as you want, instead of relying on the compiler to get it close enough to perfect before it exhausts its budget of caring about optimizing it.