Comment by ascar

If there is no dependency between your results, you do not need synchronization. Parallelizing it just means putting them into a seperate stream of execution. That can be a thread or an entirely different computer and the two different execution streams need to synchronization or communication between each other.

That is an entirely different set of problems than having to deal with a computation that has close dependencies and now you need synchronization and communication to progress the compuation. You don't only have to think about how to synchronize your computation, you also have to think about how to distribute your data to begin with to minimize the need for synchronization. An entire set of problems that just don't exist in the former case.

To pick up a word you used: In one case you have many independent tasks and you want to run them as quickly as possible. In the other case you have a single task and you think about how to split that up to make it faster.