Comment by drdaeman
6 months ago
Storing multiple tables in a single file would be trivially solvable by storing multiple Parquet files in a most basic plain uncompressed tarball (to retain ability to access any part of any file without downloading the whole thing). Or maybe ar or cpio - tar has too many features (such as support for links) that are unnecessary here. Basically, anything well-standardized that implements a very basic directory structure, with a simple index located at a predictable offset.
If any tools would've supported that.
Couldn't agree more. If tooling would just settle on an arbitrary archive format our lives would be better.