← Back to context

Comment by spacechild1

4 years ago

> In any case, the "Multithreaded Singleton" problem is devilishly difficult to write and full of subtleties.

First off, global state does not necessarily require the singleton pattern.

But let's assume that we really need it, e.g. to create a global resource only on demand. This is how it's done in C++:

  class Foo {
  public:
      static Foo& getInstance() {
          // Since C++11, local static variable initialization is thread-safe!
          static Foo instance;
          return instance;
      }
  private:    
      Foo() {
          // expensive constructor
      }
  };

I mostly use C++ and various scripting languages, but judging from the examples in https://en.wikipedia.org/wiki/Double-checked_locking it seems like C#, Java and Go all have at least one simple and safe method to achieve this.

> Global state is important in many cases and cannot be avoided, and threads absolutely complicate it even more than usual.

If you depend on global state in the parent process, how can the subprocesses even operate, since they do not have access to that state? Yes, certain resources, such as loggers, need to be global, but these should really be thread-safe anyway.

> You haven't listed off what makes threads actually easier yet.

* creating and joining threads (in a portable way!) is trivial in most programming languages; creating and joining subprocesses not so much * exchanging data much easier, no marshalling needed * logging is much easier * error handling is much easier * debugging is much easier. (How do you debug a short lived subprocess? The process terminates before the debugger even has a chance to attach to it.)

> You're saying "mutexes and queues" are easier, and I disagree.

Maybe I was not clear, I was really talking about concurrent queues. The producer can push messages, the consumer waits on messages and processes them. It works basically like a pipe/FIFO, but without the pitfalls.

> > If one thread crashes, the whole process dies. There is no consistency problem here. > Are you sure? > Lets say I pthread_cancel() one of your threads. Is that cool?

I was talking about threads crashing.

> If thread #45 accepts() a connection, then gets pthread_cancel()d,

pthread_cancel() is dangerous, I agree. Generally, you should not use it. (I have never needed it.) There are much saner ways to "cancel" a thread, depending on your language. Often it's enough to periodically check a boolean flag.

> Lets compare / contrast with kill(SIGKILL), which is still dangerous but... the state of a process is far more consistent.

Yeah, it is easy to kill a subprocess. But let's consider the opposite: what happens if the parent process dies? How do you make sure that a long running subprocess automatically terminates? It is definitely not trivial.

---

It's fine if you like subprocesses. I just wanted to challenge the notion that they are somehow easier to use than threads - which just doesn't match my experience. We are probably working in entirely different domains, so it's natural that our experiences differ.

> it seems like C#, Java and Go all have at least one simple and safe method to achieve this.

Its only simple after you've studied double-checked locking. Initial attempts often lead to failure.

> If you depend on global state in the parent process, how can the subprocesses even operate, since they do not have access to that state?

Plenty of ways to get access. The #1 way is probably to use a database to share that state in a concurrency-safe way. sqlite3 works, though postgresql is more scalable.

There are also solutions that require less resources: flock() a file and then read the shared state in a manner that's cohesive across processes. If your process dies while flock()ing something, the flock() automatically undoes (unlike mutexes where if pthread_cancelled() you could very well have a permanently locked mutex).

That's why so many systems have a database + dedicated process that handles this kind of shared global state between processes.

> Yes, certain resources, such as loggers, need to be global, but these should really be thread-safe anyway.

Loggers are an excellent example of where opening up a pipe or socket to syslogd is far easier than trying to shoehorn in a mutex+queue across threads.

> Yeah, it is easy to kill a subprocess. But let's consider the opposite: what happens if the parent process dies? How do you make sure that a long running subprocess automatically terminates? It is definitely not trivial.

If the parent of a process group is terminated, SIGHUP is sent to its children. Catch the signal then terminate.

So once again: processes handle both situations (parent dies, kill children. Or children die, notify parent), with SIGHUP and SIGCHLD respectively.

No such signaling exists in pthread_blah world. You're (trying to) argue about the "superiority" of threads when processes have all of these issues 100% figured out, while the pthread-world is completely ignorant to these issues.

  • > The #1 way is probably to use a database to share that state in a concurrency-safe way. sqlite3 works, though postgresql is more scalable.

    > There are also solutions that require less resources: flock() a file and then read the shared state in a manner that's cohesive across processes.

    Wow. And that is somehow easier than using, say, a concurrent collection? (I have never used a database in my life, so we obviously come from very different angles :-)

    > (unlike mutexes where if pthread_cancelled() you could very well have a permanently locked mutex).

    Again, there is almost never a good reason to use pthread_cancel() in the first place.

    > opening up a pipe or socket to syslogd

    In my projects, logging means "print to stderr or write to a file" :-)

    > If the parent of a process group is terminated, SIGHUP is sent to its children. Catch the signal then terminate.

    So you need to make a process group... What if your code should be cross platform? Do you know how to do this on Windows?

    > No such signaling exists in pthread_blah world

    This kind of signaling does not exist because it is not necessary. Tasks either periodically check a boolean flag or get notified via the queue itself. Here is a random simple example: https://openframeworks.cc/documentation/utils/ofThreadChanne....

    Also, I'm not sure why you keep talking about pthreads... Modern programming languages have their own (portable) threading abstractions.

    • > Wow. And that is somehow easier than using, say, a concurrent collection? (I have never used a database in my life, so we obviously come from very different angles :-)

      When it comes to understanding locks, concurrency, and parallelism with shared data between threads... yeah. In my experience, the database is heavy lifting but absolutely ensures that most issues are taken care of.

      But as I stated earlier: lighter weight solutions, such as flock() exist for a reason. If the database is too heavy (note: sqlite3 is extremely lightweight, so I bet it works for most cases), then flock() a file and read/writing to it works too.

      What flock() gets you is that 100% certainty about cleanups upon strange exit cases that threads do not get you. A flock() always cleans itself up on process termination. No guarantees about mutexes (or other issues) on thread-cancellations or other thread-related issues. (I dunno, oom killer)

      > Again, there is almost never a good reason to use pthread_cancel() in the first place.

      On the contrary. The pthread community knows that pthread_cancel() is poorly behaved and constantly tells beginner programmers not to use it.

      "There's no good reason" because everyone knows that the number of traps in using that function are legion. Its never worthwhile to use that function because it just leads to severely buggy behavior in practice.

      > So you need to make a process group... What if your code should be cross platform? Do you know how to do this on Windows?

      ... you know that Win32 doesn't support pthreads, right? And C++ std::thread doesn't support anything that we've talked about either.

      To answer your question: Win32 job objects. Every reasonable modern OS supports the concept of sessions (Windows just calls them job objects instead). The OS-level (be it session leaders / session groups in Linux, or Job objects in Windows) is the correct solution to this problem.

      EDIT: Found the function name for ya: https://learn.microsoft.com/en-us/windows/win32/api/jobapi2/...

      > This kind of signaling does not exist because it is not necessary. Tasks either periodically check a boolean flag or get notified via the queue itself. Here is a random simple example: https://openframeworks.cc/documentation/utils/ofThreadChanne....

      > Also, I'm not sure why you keep talking about pthreads... Modern programming languages have their own (portable) threading abstractions.

      And you damn well know that under a thread-kill or thread-cancel scenario, this code stops functioning. While all the code I talked above will function correctly even in the worst-case "kill -9" SIGKILL.

      Whatever underlying synchronization that channel is using to synchronize thread access will stop working if a pthread_mutex_lock() is called, but its corresponding pthread_mutex_unlock() fails to be called due to cancellation or other issue.

      7 replies →