Comment by dragontamer
4 years ago
When I call "wait" on a process, what do I know about it? I know that all fds have closed, all sockets have closed, etc. etc. A hell of a lot more than the thread example.
When I call pthread_join() on a thread, what do I know about it? Nothing. You have to assume it cleaned itself up correctly.
That's all I'm trying to point out. There's literally no assurances on pthread_join() or pthread terminations or pthread cancellations. None at all.
----------
Whether or not you wish to take advantage of this or not is your choice. If you want to just presume that pthreads always clean themselves up without any issues or subtle threading bugs to the other threads... sure. Or that we're perfect programmers who never make such mistakes...
But I have learned time and time again: if its not me who makes such a mistake, then maybe a coworker who is touching the codebase. Having 100% assurances (like the fd closures) on processes is something I can absolutely build code around without assuming the internal state or "correctness" of other people's code (including my own).
> And I'm perfectly fine with standard C++ RAII classes.
Then you should be comfortable with process cleanups and how to code around it to form strong guarantees of correctness. Enforced by Linux (even stronger than the compiler).
kill and pthread_cancel may prevent C++ destructors from running. But even kill -9 will correctly clean up processes (including the SIGCHLD signals and other such details).
Is it perfect? Of course not. But its one more assurance you lose when you go for threads instead of processes.
Sorry for being so insistent.
> When I call pthread_join() on a thread, what do I know about it? Nothing. You have to assume it cleaned itself up correctly.
I don't really understand your point. A thread just runs a function. If that function leaks resources, that's a problem with that particular function, not with threads in general. It would be just as problematic when running sequentially.
If you are worried about a particular function or module, sure, you can run it in isolation in a subprocess. But this is completely orthogonal to the topic of parallelism or threads. Just because something runs as a subprocess does not mean it will execute in parallel, it depends on how the subprocess communicates with the parent process.
I mean, if you prefer subprocesses, that's fine, but don't sell them as some kind of silver bullet.
> kill and pthread_cancel may prevent C++ destructors from running.
Sigh. As I said again and again, you wouldn't use pthread_cancel() in the first place. It's like complaining that setjmp() breaks your C++ code. Also, SIGKILL will terminate the whole process, unless caught by a signal handler.
> But even kill -9 will correctly clean up processes
Yes, it will release any OS resources, but that alone is not "correct clean up". For example, C++ destructors won't run. Sending SIGKILL is like pulling the plug on your desktop, you only do it if nothing else works.
> I mean, if you prefer subprocesses, that's fine, but don't sell them as some kind of silver bullet.
I've listed 5 different multithreaded techniques at the beginning of this discussion.
1. Processes 2. Mutexes 3. Condition Variables 4. Seq-Cst Atomics 5. Atomics + Memory Barriers
If you've thought I'm listing silver bullets, you're mistaken severely. I'm pointing out that #1 is easier than #2. And #2 is easier than #3, and #3 tends to be easier than #4. Prefer easier solutions over complex solutions.
That's all I'm trying to point out. Of course there's no silver bullets. But there are "preferred" solutions. Generally speaking, easier solutions vs harder solutions.
---------
Processes are easier because they're (partially) isolated. In contrast, threads have no isolation. Heck, one can argue that VMs and Docker are also responses to this isolation issue.
If you're unable to see why that's easier, I dunno how else to explain it. I've given it a bunch of posts.
> I've listed 5 different multithreaded techniques at the beginning of this discussion.
Unsurprisingly, I have a problem with that list as well. It mixes things that belong to different categories. A process is an execution context while the rest are synchronization mechanisms. For example, it is possible to synchronize two processes with atomic variables (in shared memory). Also, the list misses any kind of higher level threading primitive (concurrent queues, channels, futures, etc.). There is a whole world above explicit mutex locking.
> I'm pointing out that #1 is easier than #2.
I totally believe you when you say that subprocesses are the preferred solution for your particular use cases. It just does not make sense as general advice. It may be practical that subprocesses are isolated, but you cannot just downplay all the downsides.