Comment by jtbaker
15 hours ago
> the only big gotcha I encountered was finding, somewhat counter-intuitively, that it's often best minimize partitioning.
For parquet, I think with partitioning, it's really important to be mindful of the ordering of the data within the parquet file and also the query patterns of the main use cases. A little hard to generalize well to every pattern I guess.
No comments yet
Contribute on Hacker News ↗