Comment by arjie

Very cool. Okay, I think you're right. Doing dedupe at the application layer is a much better idea. I do have 512 GiB of DDR5 (it's an Epyc 9755-based server) but I think you're right because I am fully aware of the data I'm storing (internet archive data) so I can simply delta-code on a per webpage sense.

Right, I knew from /r/homelab that many normal people now store petabytes in their nodes. My specific machine is going to be in a DC located some 1 hr from me so I don't mind noise, but I am particular about power consumption and so on.

Based on what you said I'm going to run RAIDZ2 on this. I happen to have a bunch of EXOS 18 TiB drives so I shall use those. Thank you for the advice from experience!