Comment by scrp
5 days ago
After years in the making ZFS raidz expansaion is finally here.
Major features added in release:
- RAIDZ Expansion: Add new devices to an existing RAIDZ pool, increasing storage capacity without downtime.
- Fast Dedup: A major performance upgrade to the original OpenZFS deduplication functionality.
- Direct IO: Allows bypassing the ARC for reads/writes, improving performance in scenarios like NVMe devices where caching may hinder efficiency.
- JSON: Optional JSON output for the most used commands.
- Long names: Support for file and directory names up to 1023 characters.
> RAIDZ Expansion: Add new devices to an existing RAIDZ pool, increasing storage capacity without downtime.
More specifically:
> A new device (disk) can be attached to an existing RAIDZ vdev
So if I’m running a Proxmox on ZFS and NVMEs, will I be better off enabling Direct IO when 2.3 gets rolled out? What are the use cases for it?
Direct IO useful for databases and other applications that use their own disk caching layer. Without knowing what you run in Proxmox no one will be able to tell you if it's beneficial or not.
I would guess for very high performance NVMe drives.
The first 4 seem like really big deals.
The fifth is also, once you consider non-ascii names.
Could someone show a legit reason to use 1000-character filenames? Seems to me, when filenames are long like that, they are actually capturing several KEYS that can be easily searched via ls & re's. e.g.
2025-Jan-14-1258.93743_Experiment-2345_Gas-Flow-375.3_etc_etc.dat
But to me this stuff should be in metadata. It's just that we don't have great tools for grepping the metadata.
Heck, the original Macintosh FS had no subdirectories - they were faked by burying subdirectory names in the (flat filesysytem) filename. The original Macintosh File System (MFS), did not support true hierarchical subdirectories. Instead, the illusion of subdirectories was created by embedding folder-like names into the filenames themselves.
This was done by using colons (:) as separators in filenames. A file named Folder:Subfolder:File would appear to belong to a subfolder within a folder. This was entirely a user interface convention managed by the Finder. Internally, MFS stored all files in a flat namespace, with no actual directory hierarchy in the filesystem structure.
So, there is 'utility' in "overloading the filename space". But...
2 replies →
But I presume it is still not possible to remove a vdev.
That was added a while ago:
https://openzfs.github.io/openzfs-docs/man/master/8/zpool-re...
It works by making a readonly copy of the vdev being removed inside the remaining space. The existing vdev is then removed. Data can still be accessed from the copy, but new writes will go to an actual vdev while data no longer needed on the copy is gradually reclaimed as free space as the old data is no longer needed.
Although "Top-level vdevs can only be removed if the primary pool storage does not contain a top-level raidz vdev, all top-level vdevs have the same sector size, and the keys for all encrypted datasets are loaded."
3 replies →
Is this possible elsewhere (re: other filesystems)?
It is possible with windows storage space (remove drive from a pool) and mdadm/lvm (remove disk from a RAID array, remove volume from lvm), which to me are the two major alternatives. Don't know about unraid.
20 replies →
Bcachefs allows it
2 replies →
btrfs has supported online adding and removing of devices to the pool from the start
Btrfs
1 reply →
How well tested is this in combination with encryption?
Is the ZFS team handling encryption as a first class priority at all?
ZFS on Linux inherited a lot of fame from ZFS on Solaris, but everyone using it in production should study the issue tracker very well for a realistic impression of the situation.
Main issue with encryption is occasional attempts by certain (specific) Linux kernel developer to lockout ZFS out of access to advanced instruction set extensions (far from the only weird idea of that specific developer).
The way ZFS encryption is layered, the features should be pretty much orthogonal from each other, but I'll admit that there's a bit of lacking with ZFS native encryption (though mainly in upper layer tooling in my experience rather than actual on-disk encryption parts)
These are actually wrappers around CPU instructions, so what ZFS does is implement its own equivalents. This does not affect encryption (beyond the inconvenience that we did not have SIMD acceleration for a while on certain architectures).
>occasional attempts by certain (specific) Linux kernel developer
Can we please refer to them by the actual name?
1 reply →
The new features should interact fine with encryption. They are implemented at different parts of ZFS' internal stack.
There have been many man hours put into investigating bug reports involving encryption and some fixes were made. Unfortunately, something appears to be going wrong when non-raw sends of encrypted datasets are received by another system:
https://github.com/openzfs/zfs/issues/12014
I do not believe anyone has figured out what is going wrong there. It has not been for lack of trying. Raw sends from encrypted datasets appear to be fine.