Comment by ryao
5 days ago
That was added a while ago:
https://openzfs.github.io/openzfs-docs/man/master/8/zpool-re...
It works by making a readonly copy of the vdev being removed inside the remaining space. The existing vdev is then removed. Data can still be accessed from the copy, but new writes will go to an actual vdev while data no longer needed on the copy is gradually reclaimed as free space as the old data is no longer needed.
Although "Top-level vdevs can only be removed if the primary pool storage does not contain a top-level raidz vdev, all top-level vdevs have the same sector size, and the keys for all encrypted datasets are loaded."
I forgot we still did not have that last bit implemented. However, it is less important now that we have expansion.
> However, it is less important now that we have expansion.
Not really sure if that's true. They seem like two different/distinct use cases, though there's probably some small overlap.
And in my case all the vdevs are raidz