Comment by jemfinch
13 hours ago
> OS threads are expensive: an operating system thread typically reserves a megabyte of stack space and takes roughly a millisecond to create.
It's typically less than a hundred kilobytes and (on the systems I've benchmarked using std::thread) it takes 60usec (wall time in userspace) to create and destroy a thread.
Threads have gotten so fast that paying the async function coloring price makes very little sense for most software.
Its the stack space allocated to each thread that prevents you from spawning more than a thousand threads. Strategies like a thread per network connection do not scale.
I regularly spawn several thousands of threads in the C++ servers I write, and they perform well. At least 40% of FAANG companies just reduce the size of their per-thread stacks. "Thread per-connection" works just fine, and when you need to go faster, thread pools work even better without coloring all your functions.
There is an extremely small set of domains where simple threading doesn't suffice, and async/await is too high a price to pay across the entire software ecosystem just to slightly optimize those domains.