Comment by spacechild1
4 years ago
> Ease of development. Ease of maintenance. Seperation of concerns.
I'm not convinced. Managing processes is much more complex than managing threads, particularly if the code should be cross-platform. You need a very good library to hide away all the nasty OS specific details. Same goes for pipes or sockets. Then there is the whole issue of message (de)serialization. I don't see how this can possibly be easier than starting a thread and communicating with concurrent queue.
I mean, subprocesses certainly have their use cases, but I would never see them as a drop-in replacement for threads.
> you can simplify by letting the OS handle some of that.
Processes and threads are both OS resources. For the OS scheduler they are practically equivalent.
Consider: what is the easiest way to parallelize GCC to a 32 core system?
Answer: make -j128.
Way easier than trying to make every data structure inside of the compiler into a parallel programming model.
--------
Process level parallelism is a higher level of thinking. As long as you have spare RAM (and let's be frank, we all have 32GB+ sitting around these days), you can add more and more processes to solve your problem.
If you are talking data structures and mutexes, you are working at a far more complex layer than what I'm saying for #1.
----
Even completely single threaded code can be run in parallel in many cases in practice, because we have multiple files or other divisions of labor available at the process / user level.
> Consider: what is the easiest way to parallelize GCC to a 32 core system?
> Answer: make -j128.
> Way easier than trying to make every data structure inside of the compiler into a parallel programming model.
That's a false dichotomy. If the compiler program was implemented as a library, I would rather create 128 threads that each call compileSourceFile() than spawn 128 subprocesses that do the same thing.
The question you should be asking is: do I need my task to execute in a seperate address space? If yes, spawn a subprocess, otherwise use a thread.
> I would rather create 128 threads that each call compileSourceFile() than spawn 128 subprocesses that do the same thing.
Would you rather write "compileSourceFile()" in a reentrant way (ie: no global variables, no static variables, guaranteed reentrancy, and other such requirements to work in a typical pthread manner)... or would you rather have processes where all those things are fine and not bugs?
The minute you start up threads with implicitly shared memory spaces... the minute "singleton pattern" suddenly grows complex.
> The question you should be asking is: do I need my task to execute in a seperate address space? If yes, spawn a subprocess, otherwise use a thread.
On the contrary. Separate address spaces by default is far easier. Thread#45 going crazy due to buffer-overflows will demolish thread#25.
But process#45 with a buffer-overflow will not affect process#25.
I/O is also grossly simplified inside the process model. Closing out a process closes() all sockets, pipes, and file I/O automatically, no matter how the process dies. (Ex: Segfaults, kill -9, etc. etc. are all handled gracefully).
If one thread dies, for whatever reason, your program is extremely hosed. Its very difficult to reason where the legitimate state of your multithreaded data-structures is in.
Threads are far more efficient, yes. So if you need efficiency, use them. But most people in my experience are Python or PHP programmers (or other such high level language), where it is clear that performance isn't an issue.
13 replies →
> I'm not convinced.
Great! Because that's not what I'm trying to do. Figure out what works best for you. We did.
> particularly if the code should be cross-platform.
Ours, currently, does not.
> Then there is the whole issue of message (de)serialization. I don't see how this can possibly be easier than starting a thread and communicating with concurrent queue.
We develop in C#. All of this is available from MS.
> Figure out what works best for you. We did.
I did as well :-)