Comment by dmitrygr

2 days ago

x86 segmentation makes it very hard to statically analyze anything. In real mode, any byte can be referenced in 4096 different ways. It is even messier in protected mode, since now every selector is an entry in a table, so its value itself is meaningless. So, without runtime analysis, there is no way to tell if 04:1234 is or is not the same byte as fa:1204

3 comments

dmitrygr

Reply
jcranmer  2 days ago

> It is even messier in protected mode, since now every selector is an entry in a table, so its value itself is meaningless.

Actually, my experience is that things are much easier in protected mode. Since selector values are chosen by the OS, that means you rely a lot more on internal relocations. And the use of segment selectors is a strong indicator that you have a pointer in the first place.

Unfortunately, ghidra itself struggles to apply these techniques, especially in the decompiler, which seems completely unable to cope with the concept of far pointers.

  • dmitrygr  2 days ago

    In DOS, plenty of applications/games load selectors and do nasty things with them

    so indeed you'd know it is a far pointer, but may not know what to :D