Comment by JackC

Cool, I was just looking for something like this!

It's surprisingly annoying in built-in python to do something like this. The most recent thing I was trying to do was:

- load URLs from a file - hand them out to one subprocess per cpu - download them concurrently in threads or async within each subprocess - pull the results back into a single process for formatting and storing

Getting this to work and handle queues, ctrl-c, exceptions etc. is just a whole mess involving python builtins created at different times with different interfaces; I hacked until I kind of got it working, but didn't love it. Bundling it all in a single tested package would be great.