Comment by sanderjd
1 day ago
I just ran into this recently, where I had an obscure bug caused by needing to close more file descriptors in the forked process. "I want a clone of the current process" is just way less common in my experience than "I want a completely new process". It feels crazy that we don't have a way to directly express the latter thing, and can only approximate it by cloning and then fixing things up in post.
But you generally want to communicate with that process, so you do need to setup e.g. file descriptors and stuff, which needs information from the parent process to be passed.
Yes, you do want to pass in some stuff. But by default you get every single open file descriptor and a copy of every single stack that any threads use for execution.
It shares way too much, and have huge use cases where it is really, really bad.
A variant of exec could take an initial table of file descriptors in the current process that get cloned into the new child. Pipe creation could also get rolled into this mechanism. That should take care of the most obvious leaky bit of fork()/exec(), at least.
Most programming languages abstract this out to be able to connect or drop the 3 standard pipes. Typically this is the only thing that can be shared anyway unless the other program is specifically shared and expects other file handles to be available, in which case fork might be the right system call anyway.
Right. It's not that fork is useless, it's that it's weird that it's the only way to do this thing that it isn't particularly well suited for.
Keep in mind that this is the only way to start any process. Even if you just want to launch some throwaway utility program.
Nevertheless, inclusion would be a better default than exclusion in most use cases I've ever had for process spawning.
Isn't that covered by O_CLOEXEC?
I think it is error prone to need to iterate file descriptors and set this in order to inherit nothing. Excluding by default would make sense IMO.
There's a bunch of nastiness around that too. If you have e.g. library state that assumes the fd still works you can get her very confusing bugs once another file is opened into that fd number...
You may be mixing up fork and exec. Library data state isn't retained over execve(), and O_CLOEXEC does not take effect at fork().
1 reply →
>It feels crazy that we don't have a way to directly express the latter thing
Isn't that what posix_spawn is for?
posix_spawn addresses the need from userspace. Under the hood, it's still doing more or less a fork/exec, with the baggage that comes with it. A syscall would be nicer.
And how do you think posix_spawn is implemented?
This is an oft-overlooked point. An obvious place to look for improving fork+execve is to see whether posix_spawn can be given more efficient kernel mechanisms to be based upon.
And of course that has already been done. On NetBSD, posix_spawn() is a fully-fledged system call and much of the work is done in kernel mode.
* https://blog.netbsd.org/tnf/entry/posix_spawn_syscall_added
2 replies →
What do you mean by "a completely new process"?
The equivalent of CreateProcessW https://learn.microsoft.com/en-us/windows/win32/api/processt...
A process that shares nothing with the process that spawned it.
A thing that makes that complicated is that while you want that conceptually, you don't want that in reality. For instance, if the spawning process is in a container of some sort and it spawned a process that "shares nothing with the process that spawned it", the spawned process would no longer be in that container, because the state of "being in the container" is one of the things it shares with the parent process.
This is just an example of I don't even know how many things a modern-day process will share from its parent.
By "complicated" I do not even remotely mean "unsolvable". I just mean that if you really dig down into what it means to "share nothing" in a modern operating system, it's a lot richer than it was back when fork+exec was a practical solution. There's a lot of fuzzy things that could go either way when you say "shares nothing".
7 replies →
That’s how you get zombie processes and memory leaks.