Comment by esoterae
6 years ago
Sort of. But no snapshots.
Wanna use LVM for snapshots? 33% performance hit for the entire LV per snapshot, by implementation.
ZFS? ~1% hit. I've never been able to see any difference at the workloads I run, whereas with LVM it was pervasive and inescapable.
That was with the old LVM snapshots. Modern CoW snapshots have a much smaller impact. Plus XFS developers are working on internal snapshots, multi-volume management, and live fsck (live check already works, live repair to come).
I don't doubt this but do you have any documentation?
Asking for a friend who uses XFS on LVM for disk heavy applications like database, file server, etc.
You would have to look at the implementation directly. The user documentation isn't great for documenting performance considerations, sadly.
Essentially it comes down to this: a snapshot LV contains copies of old blocks which have been modified in the source LV. Whenever a block is updated in the source LV, LVM will need to check if that block been previously copied into all corresponding snapshot LVs. For each source LV where this is not the case, it will need to copy the block to the snapshot LV.
This means that there is O(n) complexity in the checking and copying. And in the case of "thin" LVs, it will also need to allocate the block to copy to, potentially for every snapshot LV in existence, making the process even slower. The effect is write amplification effectively proportional to the total number of snapshots.
ZFS snapshots, in comparison, cost essentially the same no matter how many you create, because the old blocks are put onto a "deadlist" of the most recent snapshot, and it doesn't need repeating for every other snapshot in existence. Older snapshots can reference them when needed, and if a snapshot is deleted, any blocks still referenced are moved to the next oldest snapshot. Blocks are never copied and only have a single direct owner. This makes the operations cheap.
FreeNAS has good documentation on which hardware to pick and how to set up ZFS.
That's for the old "fat" LVM snapshots, right ? No way the new CoW thin LVs have such a big overhead for snapshots.
There will be a much bigger overhead in accounting for all of the allocations from the "thin pool".
The overlying filesystem also lacks knowledge of the underlying storage. The snapshot much be able to accommodate writes up to and including the full size of the parent block device in order to remain readable, just like the old-style snapshots did. That's the fundamental problem with LVM snapshots; they can go read-only at any point in time if the space is exhausted, due to the implicit over-commit which occurs every time you create a snapshot.
The overheads with ZFS snapshots are completely explicit and all space is fully and transparently accounted for. You know exactly what is using space from the pool, and why, with a single command. With LVM separating the block storage from the filesystem, the cause of space usage is almost completely opaque. Just modifying files on the parent LV can kill a snapshot LV, while with ZFS this can never occur.