Comment by OscarCunningham
3 years ago
So this might be a naive question, but is it literally that you're simulating a fluid in parallel by having each thread simulate a portion of the space? And then the message passing between threads is the data about the fluid on the boundary?
Depending on the method, but generally yes, the physical domain is decomposed into the number of cores. Data is synchronized at processor boundaries at a very high rate to maintain consistency.
Computational fluid dynamics is actually a problem where parallelization doesn't get you much. It is primarily limited by memory bandwidth.
I am not sure where you get this information. Parallelization is everything in this space, hence why we have highly interconnected supercomputers to model the most difficult engineering problems. Typical runs use 30k+ cores for a single problem for weeks on end [0]. There are some special cases, such as Boltzmann/dvm [1] where invididual partitions of cells have millions of degrees of freedom, where memory bandwidth is the primary concerns. Even then, doing domain decomposition to a larger number of cores takes care of the issue.
[0] https://www.nas.nasa.gov/SC22/research/project12.html
[1] https://www.sciencedirect.com/science/article/abs/pii/S00219...
Those large number of cores come with a diminishing return (see this benchmark, for example: https://nusit.nus.edu.sg/services/hpc-newsletter/cfd-simulat...)
1 reply →
It's very, very hard to be memory bandwidth bottlenecked if your program is not parallelized, even on consumer hardware. A 5Ghz cpu core with dual-channel DDR5 might get 75 GB/s of memory bandwidth. That's 15 bytes per clock cycle. Prosumer hardware? Maybe 60 bytes per clock cycle. No way one cpu core can keep up with that.
Memory bandwidth can be increased by parallelization though. E.g. MPI (Message Passing Interface) is one of the major libraries in parallel programming and supercomputing and deals with parallelization across multiple machines.