Comment by dzaima

That'd only help for one object per address space. Main thing needing relocation - shared libraries - needs arbitrarily-many segment bases.

And when you're not a library, relocation is just a mild probabalistic security improvement (...that'd be massively-more bypassable than it already is if the program was littered full of gadgets of "read register as unrelocated offset and use it with its correct base" instructions).