Comment by ralferoo

Maybe answering my own question, but I'm now wondering if the reason that the divide by 4 was chosen (so essentially using byte addressing instead of word addressing) is so that the linker can do symbol fixup / relocation.