Comment by Kalanos

Hadoop hasn't been relevant for a long time, which is telling.

Unless I had thousands of files to work with, I would be loathe to use cluster computing. There's so much overhead, cost, waiting for nodes to spin up, and cloud architecture nonsense.

My "single node" computer is a refurbished tower server with 256GB RAM and 50 threads.

Most of these distributed computing solutions arose before data processing tools started taking multi-threading seriously.