Comment by ocd
10 hours ago
I'm sure people have reasons for taking these things as far as they do with ZFS, and everything else that goes into what is commonly considered a "NAS." But I've found great convenience in a tight NFSv3 config running from a single high capacity HDD with ext4 on a Linux machine (primary system that's always on anyway while I'm around/awake) making things available to my other devices for the electricity and overhead cost of one machine instead of multiple.
I try to keep my network configuration restrictive by default, so I'm not too concerned about possible security arguments running it from my main machine. I've probably committed some great sin here, but is plain NFSv3 and secondarily Samba (for compatibility) really not enough?
First: Do you have backups of your single high-capacity HDD? That's my biggest worry. What's your plan for when that HDD fails one day, as it will?
The main reasons people go for ZFS and a "NAS" is checksums and data integrity protection, as well as maybe not wanting to keep their primary machine awake all the time (I personally don't).
Then there are useful features like snapshots, which means I don't have to worry as much about accidentally deleting, or over-writing a file and losing data.
I don't see anything wrong with using a main machine that's up 7x24 as a NAS, don't buy things for the sake of something, but I'm worried about your reliability and bitrot protection. (Yes, it happens, I've seen it first-hand thanks to ZFS).
I have external parity for everything with rsync. I don't have any storage that isn't matched with external backups. Fortunately I've never seen any bitrot or had a drive fail on me since I tend to replace them before they do. And yeah, I could add in some extra HDDs for RAID, but this is more a personal choice because of how much I hate SATA.
Filesystems like ext4 are because I value boring and rock solid stability over semi-experimental status of modern filesystems with all the features.
What do you use for external parity? Par2?
If you were to replace your current drive with a new larger one and your RAM or your SATA controller silently corrupted the files how would things play out? Would these corrupted files propegate to the backups?
Ext4 is battle tested so I understand your reasoning. I think you just need to figure out a way to detect silent corruption and a way to snapshot your files in case they do get corrupted.
From experience I can tell you that it is an absolute pain having to manually sort through a bunch of files trying to detect which ones are corrupt and which ones are good.
1 reply →
How do you know you haven't seen bitrot? What rsync arguments do you use to error out when an old, untouched file suddenly changes?
1 reply →
What do you dislike about SATA?
1 reply →
rsync does not protect you from bitrot unless you have some script that specifically collects the rsync log and tracks when existing files change, and you somehow know every file that should not have changed and get alerted that a file changed when it shouldn't have.
Even with that, that is still just 2 equally authoritative copies, without any way to know which one suffered the bitrot.
Ok the tie-breaker data could come from history. Yesterdays log and the day before gives you the extra data points to say which side changed. As long as the log itself is somehow above reproach.
Except this script is a myth anyway. You would still need to have something where you tag files as "this file shall never change again, so if it ever appears to, tell me so I can ok or reject the change." and you would have to actually do that tagging and reviewing.
It's more the single drive storage layout that is an issue in your case. No resilience to a single disk failing and you are risking all your data if your hardware starts silently corrupting data. This is assuming all your backups are done using the data on the single hard drive.
If you want to keep your setup simple I would consider the following:
- Keep a list of file hashes and check your files against this list periodically.
- Use some type of backup program that supports snapshots. Restic is a good choice.
- Get a hard drive the same size as the one you have and use SnapRAID to manage file integrity or set up a two disk mirror using btrfs.