Comment by lavela
15 hours ago
"Gzip only provides a compression ratio of a little over 1000: If I want a file that expands to 100 GB, I’ve got to serve a 100 MB asset. Worse, when I tried it, the bots just shrugged it off, with some even coming back for more."
https://maurycyz.com/misc/the_cost_of_trash/#:~:text=throw%2...
Ah cool that site's robots.txt is still broken, just like it was when it first came up on HN...
You could try different compression methods supported by browsers like brotli.
Otherwise you can also chain compression methods like: "Content-Encoding: gzip gzip".