Comment by RossBencina
4 days ago
> This is more work than going through GWLP_USERDATA
Indeed, aside from a party trick, why build an executable trampoline at runtime when you can store and retrieve the context, or a pointer to the context, with SetWindowLong() / GetWindowLong() [1]?
Slightly related: in my view Win32 windows are a faithful implementation of the Actor Model. The window proc of a window is mutable, it represents the current behavior, and can be changed in response to any received message. While I haven't personally seen this used in Win32 programs it is a powerful feature as it allows for implementing interaction state machines in a very natural way (the same way that Miro Samek promotes in his book.)
[1] https://learn.microsoft.com/en-us/windows/win32/api/winuser/...
There's an annoying corner case when using SetWindowLongPtr/GetWindowLongPtr() -- Windows sends WM_GETMINMAXINFO before WM_NCCREATE. This can be worked around with a thread local, but a trampoline inherently handles it. Trampolines are also useful for other Win32 user functions that don't have an easy way to store context data, such as SetWindowsHookEx(). They're also slightly faster, though GetWindowLongPtr() at least seems able to avoid a syscall.
The code as written, though, is missing a call to FlushInstructionCache() and might not work in processes that prohibit dynamic code generation. An alternative is to just pregenerate an array of trampolines in a code segment, each referencing a mutable pointer in a parallel array in the data segment. These can be generated straightforwardly with a little template magic. This adds size to the executable unlike an empty RWX segment, but doesn't run afoul of any dynamic codegen restrictions or require I-cache flushing. The number of trampolines must be predetermined, but the RWX segment has the same limitation.
I wasn't aware of the thread local trick, I solve this problem by not setting WS_VISIBLE and calling SetWindowPos & ShowWindow after CreateWindow returns (this solves some other problems as well..)
FlushInstructionCache isn't needed on x86_64. I-cache and D-cache are coherent.
I'm not convinced this is always guaranteed for a Windows x64 program. When running on bare x64 hardware, FlushInstructionCache() does seem to be an (inefficient) noop on Windows 11 x64, but when running in emulation on Windows 11 ARM64, it's running a significantly larger amount of ARM64 native code -- it looks like it might be ensuring that stale JIT code is flushed.
The combination `GWLP_USERDATA` to pass state and `GWLP_WNDPROC` to update the actual wnd procedure is what I used in my [rust wrapper](https://github.com/timokroeger/winmsg-executor/blob/main/src...).
This two step approach is the only way I found to use rust closures for wndproc without double allocation and additional indirection.