Comment by alexrustic
1 year ago
I was asked hours ago by a concurrency enthusiast, whose website has been a valuable source of information for me on the topic, to tell in a sentence what capability Asyncpal provides to users above the Python standard library (stdlib).
I found the question interesting because it goes straight to the point and calls for a concise answer. I believe the answer is missing from this 'Show HN'. Here is my response to the question:
Asyncpal unifies the stdlib (concurrent.futures + multiprocessing.pool) and provides true elastic pools (grow + shrink) with an intuitive interface.
If I'm given a chance to develop, I would start with the last part of the sentence that might sound subjective. For example, the 'Future' class in 'concurrent.futures' exposes a method named 'result' to collect the result of a task. In contrast, the 'Future' class in Asyncpal exposes a 'collect' method and a 'result' property.
The stdlib's pools only grow in size and do not shrink, making them non-elastic. Therefore, I would not want to keep them alive in the background for sporadic workloads [1].
Discussions on 'concurrent.futures' vs 'multiprocessing.pool.Pool' highlight that each has unique features. While 'concurrent.futures' is the modern package, it omits some niceties found in 'multiprocessing.pool.Pool'. For example, 'concurrent.futures' has only one 'map' [2] method, which works eagerly and therefore not suitable for very long iterables [3][4]. However, I acknowledge the superiority of 'concurrent.futures.Future' [5] over 'multiprocessing.pool.AsyncResult' [6] because tasks cannot be cancelled with the latter (among other things).
[1] https://www.cloudcomputingpatterns.org/unpredictable_workloa... (related)
[2] https://docs.python.org/3/library/concurrent.futures.html#co...
[3] https://docs.python.org/3/library/multiprocessing.html#multi...
[4] https://docs.python.org/3/library/multiprocessing.html#multi...
[5] https://docs.python.org/3/library/concurrent.futures.html#fu...
[6] https://docs.python.org/3/library/multiprocessing.html#multi...