Comment by 1vuio0pswjnm7

4 hours ago

Using RePair for compression I can also search inside compressed tarballs full of logs

To do this, I first insert a blank line at the top of each log file before adding to the tarball

IME, RePair is faster than compressing with zstd and the size reduction is almost the same

The only "catch" is that RePair requires more memory during compression