Comment by spacechild1
4 years ago
> The #1 way is probably to use a database to share that state in a concurrency-safe way. sqlite3 works, though postgresql is more scalable.
> There are also solutions that require less resources: flock() a file and then read the shared state in a manner that's cohesive across processes.
Wow. And that is somehow easier than using, say, a concurrent collection? (I have never used a database in my life, so we obviously come from very different angles :-)
> (unlike mutexes where if pthread_cancelled() you could very well have a permanently locked mutex).
Again, there is almost never a good reason to use pthread_cancel() in the first place.
> opening up a pipe or socket to syslogd
In my projects, logging means "print to stderr or write to a file" :-)
> If the parent of a process group is terminated, SIGHUP is sent to its children. Catch the signal then terminate.
So you need to make a process group... What if your code should be cross platform? Do you know how to do this on Windows?
> No such signaling exists in pthread_blah world
This kind of signaling does not exist because it is not necessary. Tasks either periodically check a boolean flag or get notified via the queue itself. Here is a random simple example: https://openframeworks.cc/documentation/utils/ofThreadChanne....
Also, I'm not sure why you keep talking about pthreads... Modern programming languages have their own (portable) threading abstractions.
> Wow. And that is somehow easier than using, say, a concurrent collection? (I have never used a database in my life, so we obviously come from very different angles :-)
When it comes to understanding locks, concurrency, and parallelism with shared data between threads... yeah. In my experience, the database is heavy lifting but absolutely ensures that most issues are taken care of.
But as I stated earlier: lighter weight solutions, such as flock() exist for a reason. If the database is too heavy (note: sqlite3 is extremely lightweight, so I bet it works for most cases), then flock() a file and read/writing to it works too.
What flock() gets you is that 100% certainty about cleanups upon strange exit cases that threads do not get you. A flock() always cleans itself up on process termination. No guarantees about mutexes (or other issues) on thread-cancellations or other thread-related issues. (I dunno, oom killer)
> Again, there is almost never a good reason to use pthread_cancel() in the first place.
On the contrary. The pthread community knows that pthread_cancel() is poorly behaved and constantly tells beginner programmers not to use it.
"There's no good reason" because everyone knows that the number of traps in using that function are legion. Its never worthwhile to use that function because it just leads to severely buggy behavior in practice.
> So you need to make a process group... What if your code should be cross platform? Do you know how to do this on Windows?
... you know that Win32 doesn't support pthreads, right? And C++ std::thread doesn't support anything that we've talked about either.
To answer your question: Win32 job objects. Every reasonable modern OS supports the concept of sessions (Windows just calls them job objects instead). The OS-level (be it session leaders / session groups in Linux, or Job objects in Windows) is the correct solution to this problem.
EDIT: Found the function name for ya: https://learn.microsoft.com/en-us/windows/win32/api/jobapi2/...
> This kind of signaling does not exist because it is not necessary. Tasks either periodically check a boolean flag or get notified via the queue itself. Here is a random simple example: https://openframeworks.cc/documentation/utils/ofThreadChanne....
> Also, I'm not sure why you keep talking about pthreads... Modern programming languages have their own (portable) threading abstractions.
And you damn well know that under a thread-kill or thread-cancel scenario, this code stops functioning. While all the code I talked above will function correctly even in the worst-case "kill -9" SIGKILL.
Whatever underlying synchronization that channel is using to synchronize thread access will stop working if a pthread_mutex_lock() is called, but its corresponding pthread_mutex_unlock() fails to be called due to cancellation or other issue.
> On the contrary. The pthread community knows that pthread_cancel() is poorly behaved and constantly tells beginner programmers not to use it.
Now tell me: how should it behave?
> Its never worthwhile to use that function because it just leads to severely buggy behavior in practice.
Well yeah. That's why we don't use it. There is no possible sane way to implement this safely. But that's not the point. Cancelling a thread is like pulling the plug on your PC. It's obviously not the right way to stop a thread. What you should do - and everybody is doing in practice - is telling the code in the thread to return. Just like you ask your OS to gracefully shutdown your PC. In the example I gave above (ofThreadChannel) this is as simple as calling the close() method.
(In the same way, sending SIGKILL isn't the proper way to stop a subprocess either. It is the last resort.)
> ... you know that Win32 doesn't support pthreads, right?
There are in fact pthread implementations/wrappers for Windows, e.g. libwinpthread from the mingw64 project. pthreads is short for POSIX threads, it is not tied to a particular OS.
> And C++ std::thread doesn't support anything that we've talked about either.
You mean something like pthread_cancel? Of course it does not. We do not need it.
> To answer your question: Win32 job objects.
The question was more rhetoric... but thanks :-)
> And you damn well know that under a thread-kill or thread-cancel scenario, this code stops functioning. While all the code I talked above will function correctly even in the worst-case "kill -9" SIGKILL.
You are right that interrupting a thread can be desastrous. However, I never ever needed to do this... and I have written a lot of multi-threaded code.
> Whatever underlying synchronization that channel is using to synchronize thread access will stop working if a pthread_mutex_lock() is called, but its corresponding pthread_mutex_unlock() fails to be called due to cancellation or other issue.
What other issues? Never had this problem... neither do all the heavily multi-threaded programs I use daily.
You somehow try to paint threads as fragile based on some obscure pthreads feature that almost nobody uses in practice...
> There are in fact pthread implementations/wrappers for Windows, e.g. libwinpthread from the mingw64 project. pthreads is short for POSIX threads, it is not tied to a particular OS.
I've had poor experience with MingW's implementation of pthreads. I've always preferred Window's native threads instead if I were on Win32. Yes, it means rewriting pthread_create code from Linux into CreateThread in Win32, but its better than the alternative.
There's just a whole bunch of "Win32-isms" that don't really make sense with how Linux assumes pthreads to work.
I'm glad that C++ std::thread exists now and is my preference these days.
> You somehow try to paint threads as fragile based on some obscure pthreads feature that almost nobody uses in practice...
Tell me. Do you use RAII?
All I'm trying to point out is that processes are RAII for _almost every known resource_ in your program.
No need to "pthread_cleanup_push" or pop those cleanup handlers. No need to figure out where pthreads could fail under cancellation points or other such obscure error conditions.
When a process exits, all FDs are closed, SIGHUP / SIGCHLD are sent to the awaiting processes as expected, and all sorts of well specified cleanup occurs.
In pthread-land, its 100% manual. You have to identify every single case and properly use them (and properly pthread_cleanup_push) to RAII the codebase into a clean state.
-------
If you've never had such cleanup issues in multithreaded code... then I hope you never come across it. But in my experience, the additional "free RAII" factor of Linux (or Windows) that is given to a full process (instead of a Thread) really improves the reliability of my programs.
So unless I have to use Threads (and I'm very well acquainted with the tools available in Thread space), I prefer using those RAII-like process cleanup functions.
God forbid an exception has an exception inside of itself in one of your threads and causes a termination condition you weren't expecting (std::terminate), or some other obscure case happens. The "free cleanup" on processes handle these sorts of obscure deaths much better than threads handle it.
5 replies →