Comment by Nathanba

Parallelism and concurrency are different models. It is not a needless distinction.

With just concurrency, you can have a system, e.g. an event processing system, where each event happens, is processed to completion, and then the next event is processed. Events can be of different types and can have different handlers. Events can be arrive (or be delivered) in different orders. The context-switching points are known (in this case, the beginning and end of event processing). As such, any computation between context switching points appears to happen atomically (i.e. no partial update, no internal reordering).

Parallelism is a different model.

There are no well-defined, limited set of interaction points between computations. A computation could literally be interrupted at any machine-level instruction and another computation start running. Suddenly, the number of possible interactions is huge--almost beyond comprehension. Locks, transactions, or other synchronization mechanisms are necessary in order to create larger atomic regions (and defend against race conditions). Worse, with weak memory models, which most modern multi-core hardware have, means that interleaving alone is not enough to explain the possible interactions between threads. It's possible with weak memory models that writes from another computation appear in different orders to different threads. Weak memory models make avoiding race conditions absolutely paramount.

> In all cases you want the maximum performance.

Not necessarily true. In many cases of concurrent programming, you don’t actually care about performance, you care about multiple different things happening seemingly at the same time (sure faster is nice, but not stuttering or freezing is more important). For example on a single core multitasking system (either in the past or on an embedded system today), you want multiple different tasks all running together, but this is achieved by giving them all a little chunk of execution one after the other — time slicing. The tasks are running concurrently, they all make progress together, but they share the same single thread of execution, they don’t happen in parallel. Yet one doesn’t block another, which is the important thing here.

I mean may e you “want” maximum performance, but that’s not what’s most important. Not having one task block another is what’s important.

Parallel programming is pretty much always about making things faster, by running then in parallel at the same time. The main benefit of doing that is that they both complete faster.

If there is no dependency between your results, you do not need synchronization. Parallelizing it just means putting them into a seperate stream of execution. That can be a thread or an entirely different computer and the two different execution streams need to synchronization or communication between each other.

That is an entirely different set of problems than having to deal with a computation that has close dependencies and now you need synchronization and communication to progress the compuation. You don't only have to think about how to synchronize your computation, you also have to think about how to distribute your data to begin with to minimize the need for synchronization. An entire set of problems that just don't exist in the former case.

To pick up a word you used: In one case you have many independent tasks and you want to run them as quickly as possible. In the other case you have a single task and you think about how to split that up to make it faster.