You might not necessarily need even shared memory - it's possible to pass a file descriptor over a socket on all modern unices (and albeit with a completely different API also on Win32) so your control process could do an accept(), maybe read the headers in there, then talk to module processes to determine what the desired approach is, and then hand over the http socket so the module process can do whatever it needs to with it.
When I imagine how I'd use this in my head the imagined design rapidly gets much more complicated than what you're currently doing, and I'm not at all arguing that that complexity is necessarily worth it ... but there's also all sorts of cool+weird things you could implement that way that would be exceedingly tricky otherwise, so I figured I'd point it out anyway :D
(the example of code that does the relevant magic to get fds across a socket that immediately springs to mind is https://fastapi.metacpan.org/source/MLEHMANN/IO-FDPass-1.3/F... - yes, warning, it's inside a perl extension, but I see no reason that would impede you borrowing the C parts if it was useful ;)
Initially I did use fork a lot to allow each handler in its own “process” mainly because I wanted to isolate it as well in a container fashion. However the overhead became to much at the start when I just wanted it to work. Will be looking into it again!
I think some stuff will be happier in the "director" process and some happier in its own "worker" process - and fd passing is the magic trick that lets you accept() in the former and still do the bulk of the work in an appropriate instance of the latter.
I 100% get the 'overhead' part - but at some point hopefully you'll have enough other stuff already running that the 'fun' factor of enabling that will win out :D
You might not necessarily need even shared memory - it's possible to pass a file descriptor over a socket on all modern unices (and albeit with a completely different API also on Win32) so your control process could do an accept(), maybe read the headers in there, then talk to module processes to determine what the desired approach is, and then hand over the http socket so the module process can do whatever it needs to with it.
When I imagine how I'd use this in my head the imagined design rapidly gets much more complicated than what you're currently doing, and I'm not at all arguing that that complexity is necessarily worth it ... but there's also all sorts of cool+weird things you could implement that way that would be exceedingly tricky otherwise, so I figured I'd point it out anyway :D
(the example of code that does the relevant magic to get fds across a socket that immediately springs to mind is https://fastapi.metacpan.org/source/MLEHMANN/IO-FDPass-1.3/F... - yes, warning, it's inside a perl extension, but I see no reason that would impede you borrowing the C parts if it was useful ;)
Initially I did use fork a lot to allow each handler in its own “process” mainly because I wanted to isolate it as well in a container fashion. However the overhead became to much at the start when I just wanted it to work. Will be looking into it again!
I think some stuff will be happier in the "director" process and some happier in its own "worker" process - and fd passing is the magic trick that lets you accept() in the former and still do the bulk of the work in an appropriate instance of the latter.
I 100% get the 'overhead' part - but at some point hopefully you'll have enough other stuff already running that the 'fun' factor of enabling that will win out :D