Comment by rsecora

3 years ago

I use it for data transformation, cleanup and enrichment. (TXT, CSV, Json, XML, database) to (TXT, CSV, JSON, XML, database).

Speed up of 30x - 40x. The highest speedup on those that require logic in the transformation. (lot of function calls, numerical operations and dictionary lookups).

2 comments

rsecora

Reply
captn3m0  3 years ago

Similar. I was working on some ETL work with SQLite, and now PyPy is my regular tool for getting better performance at similar jobs.

  • sitkack  3 years ago

    Same. I have used it for many ETL jobs, usually with about a 10x speed up. It also pulled in the latency on some Flask rest apis.