Comment by crabbone
3 years ago
I don't know who invented this nonsense distinction. First time I was introduced to this idea of "concurrent programming" being a separate thing when Go was released. So, I associate this nonsense with Go, but it could have happened earlier, I simply never heard about it before then.
Anyways. The way I see it used today, it's applied to language runtimes incapable or severely crippled when it comes to parallel / concurrent execution. Eg. Python, JavaScript etc. In such environments programmers are offered a mechanism that has many downsides of parallel / concurrent programming (eg. unpredictable order of execution) without the benefits of parallel / concurrent programming (ie. nothing actually happens at the same time, or only sleep is possible at the same time etc.)
I feel like this distinction, while a nonsense idea at its core, became so popular due to the popularity of language runtimes with disabilities and their users needing to validate their worth by adding features to their languages their runtimes are inherently incapable of implementing.
Similar situation happened with ML-style types. Python, for example, works very poorly with this add-on, but the desire to match features of other languages led Python developers to add those types anyways. Similarly, TypeScript and a bunch of similar languages, especially in Web.
How is this nonsense or anything to do with "language runtimes with disabilities"? An OS running on a single core processor cannot be parallel but it may be concurrent: it can never physically do two things at the same time, but it might be able to logically interleave different tasks.
Parallelism is a physical thing, concurrency is a logical thing.
Parallelism is a physical thing, concurrency is a logical thing.
Fundamentally the difficulty is all about synchronization. People can try to split hairs and say there are two terms for two different things but ultimately it doesn't matter because the underlying problem is the same.
Single core concurrency doesn't have to deal with hardware memory synchronization.
6 replies →
You have just repeated the nonsense I was talking about.
The claim you repeat is meaningless. A program is either parallel / concurrent or not. The situation you describe (when there's a single processor core) isn't parallel or concurrent. In some sense, it emulates concurrent / parallel execution because it imitates the unpredictable ordering of code execution, which sure has its uses... but the whole point of dealing with this unpredictable ordering is that we actually want parallelism / concurrency. The emulation on its own is worthless.
No it's not? How do you characterize running two programs on a single core without calling it concurrent but not parallel? There is a clear distinction between logical multitasking and physical multitasking that I think is useful to taxonomize.
bruh your OS scheduler wants a word with you
a program (as written) is either concurrent or not, a program (as executed) is either parallel or not
a program which is not written as concurrent can never be executed as parallel
a program which is written as concurrent can be executed as parallel, or not
> The situation you describe (when there's a single processor core) isn't parallel or concurrent.
a program running on a single core can never be parallel, but it can be concurrent
concurrent is a logical property, parallel is a physical property
2 replies →
Yes, concurrency on a single core CPU is simply not possible. That's why multitasking OS's didn't exist until multicore CPUs became a thing.
If only these "Gophers" knew anything about computing history!
2 replies →
Your "single core" computer contains a lot of other concurrently operating hardware like storage and network cards.
>> don't know who invented this nonsense distinction ...
It not nonsense. In C or C++ lots of code can be made parallel using OpenMP and inserting some #pragma statements above for loops. This does not work for things like running a UI in one thread and some other work in another thread, perhaps displaying results as they are found. These are quite different types of parallelism.
OpenMP versus std::thread/pthread/etc isn't this distinction. Both are thread-based parallelism; that OpenMP implementations do so cleverly via a threadpool approach is secondary. Any loop you can write using OpenMP you could dispatch manually via an explicit thread-based approach, and long-running parallel task you'd normally implement via std::async/std::future/std::thread could be dispatched via OpenMP. There are good conventions about what kinds of operations are better expressed via one or the other mechanism, but they are effectively equivalent in capability.
What does this have to do with anything?
I mean, great... you discovered a somewhat useful library: OpenMP... so what? How does this factoid affect the validity of the definition of code parallelism?
I think it's nonsense to invent new terminology for this tiny minor difference in how you use threads and whether you wait on results or you don't wait.
I'm sorry but you are betraying a myopic view here. Concurrency as a computer science concept has existed at least since the existence of co-routines which have been around for 60+ years. Parallelism as a concept has also existed for about as long. I find it fascinating that you take issue with "concurrency" when that one is the older concept in the engineering zeitgeist. If anything, parallelism is the odd one because it only became sounding average engineers started thinking about when consumer CPUs started becoming multi threaded.
but it's just not a tiny minor difference. The problems you end up dealing with are entirely different.
The terminology used might be terrible since they're originally synonyms. Differentiating between the two is very useful though.
7 replies →
Concurrency didn’t necessarily require threads. Concurrency is not a difference in how you use threads, its logical computer science concept. Parallelism is a physical property and parallelism is almost always concerned with doing things a faster by doing them in parallel, concurrency can be done for other purposes (eg to not have long running tasks block work). All parallel programming is concurrent but not all concurrent programming is parallel.
The distinction matters because the focus is different: they’re different topics. With concurrent programming you deal with synchronisation and topics like lock-freedom and wait-freedom. In parallel programming the main focus is about work distribution and scheduling to process it faster.
I'm not sure I can identify when it started, but these were already the concepts commonly in use when I did my CS undergraduate work in the early 90s. I.e. it was in textbooks and course titles as established jargon.
Concurrency was the kind of thing worried about in OS design or Unix programming styles whether on a time-sharing system or some small scale multi-processing system. Coordination of heterogeneous sequential programs on some shared resources.
Parallelism was the topic of high-performance computing with combined use of many hardware resources to accelerate a single algorithm.
Of course these are simplifying abstractions, and real systems can get into the murky gray area that is both concurrent and parallel.
Rob Pike has a discussion on the distinctions between parallelism and concurrency. Concurrency is closely related to co-routines which are a distinct invention from threads/processes which are more related to parallelism.
Just because you didn't know about the concept doesn't mean the distinction is nonsense. I think they are similar but not the same, exactly for the reasons laid out in the comment you are replying too. Just because you hate the languages that support concurrent programming doesn't mean concurrent programming is meaningless. Any language using async/await (basically all of them these days) support concurrent programming, including languages such as Swift and C# which are nothing like Python or JavaScript.
> In such environments programmers are offered a mechanism that has many downsides of parallel / concurrent programming (eg. unpredictable order of execution) without the benefits of parallel / concurrent programming (ie. nothing actually happens at the same time, or only sleep is possible at the same time etc.)
From the developer's perspective it's a massive upside to not have to manage low-level details and just define how the event loop will call their code.
Any modern web browser has plenty of parallel execution behind the scenes, but the developer (and user) will just see concurrency which is much simpler to reason about. The order of execution doesn't matter if the things being executed aren't dependent. What matters more is that there's only one thread to think about. If they are dependent they shouldn't have been parallelized in the first place, so they're not.
What does this have to do with low or whatever other level?
[dead]