Comment by jjav

> "email Jeff and ask him to run his script" isn't scalable

Sure, it's not.

But the only alternative to that is not building some monster cluster to process a few gigabytes.

You can write a good script (instead of hacking one together), put it in source control and pull it from there automatically to the production server and run it regularly from cron. Now you have your robustness, reproducibility and consistency as well as much higher performance, for about one-ten-thousandth of the cost.