Comment by rao-v
4 hours ago
I remember being so mad years ago, coming from a pure CS background, when it dawned on me that async await was “mere” control flow and not actual parallelism.
It’s why I feel go (with go routines being the norm) is one of the few imperative languages that was designed vs. filling out a bunch of historical constraints (apologies this is not meant to trigger a language debate, just an idiosyncratic thought)
Being mere control flow is a good thing: some async/await implementations are just desugared into a state machine anyways, and it totally works on a single thread. Early async/await in Python was just a small generalization of its existing generator mechanism, and nobody would think generators in Python enables parallelism: it was always a control flow construct. This cleanly demonstrates the separation between the concepts of concurrency versus parallelism.
And of course go routines and channels can also be desugared into mere control flow. That’s how ClojureScript does async.
> vs. filling out a bunch of historical constraints
Do you mind elaborating on this? I don't understand what you're trying to get at.
Threads were historically expensive enough that “just spawn a thread” wasn’t a reasonable thing to do in many situations. Thread pools were sort of a last resort, and we ended up with control flow like objects (futures, await etc.) to multiplex concurrency without parallelism.
Go sort of asks why tho and just standardizes on go routines as a good abstraction over both concurrency and parallelism.
This is sorta true elsewhere too. Go rejects a lot of the machinery that OO languages seem to feel obliged to carry around - inheritance hierarchies, explicit interface implementation etc. For what it's worth, I don't write much go, and I don't think it's magical. I just like how clearly it revisited some basics.
Elsewhere on HA you’ll find my extended rant about how strange it is that we don't have a language that elegantly abstracts computation over threads, SIMD, GPUs etc. Compilers can do this sort of thing now, just not optimally.
> Elsewhere on HA you’ll find my extended rant about how strange it is that we don't have a language that elegantly abstracts computation over threads, SIMD, GPUs etc. Compilers can do this sort of thing now, just not optimally.
Autoparallelization has been a hot topic for literally decades, quite possibly longer than you've been alive.
The problem is that the techniques you need to do to write good SIMD code versus good GPU code versus good multithreaded code versus distributed computation are all different. Taking just memory concerns: a SIMD code needs you to carefully arrange memory so that every thread is accessing an adjacent memory location. GPU code likes locality, but you have large group sizes that can share all the local memory pretty cheaply, and loading from global memory to local memory is relatively expensive, so now you have to do a lot of tuned blocking. With multithreaded code, you now want to avoid sharing between different threads (which generally requires distributing loop iterations among threads very differently). And with a distributed platform, now you're primarily worrying about the overhead of communication of data between different nodes, and you're trying to minimize that.