OpenZFS
4 HN mentions · mentioned in 4 months, 2014–2023
1:25:25 · Oct 11, 2013
An OpenZFS office hours session with ZFS co-creator Matt Ahrens answering questions on topics such as hole birth time, send/receive buffers, device removal, block pointer rewrite and performance with many snapshots.
Cited as the authoritative explanation of why block pointer rewrite is so hard to implement in ZFS.
4 comments from Hacker News
On my system the root account is primarily for upgrades, but also storage administration! ZFS snapshots protects me against silly mistakes like strange permutations of `rm -rf ~/` in day to day operations. Not having root all the time protects me against `zfs destroy -r nagato/data/home/drbawb` which will nuke my home folder, 144 frequent snapshots (15 minute intervals per 36 hours), 14 daily snapshots, 4 weekly snapshots, and 3 monthly snapshots. That would suck. A lot. (I have a file called "RIP", where a drive image used to be mounted, because I got a little too over-zealous with the recursive option on some ZFS commands on a file that had `copies=1`. _RAID is not a backup, snapshots are not a backup, etc._) This also means that theoretically ransomware (running without root) could encrypt my home directory, but apart from doubling the number of blocks used on disk it would have no real impact on my data. (If someone writes a virus that can encrypt the blocks inside a ZFS partition, without mangling ZFS data structures beyond repair, then I'll be more impressed than angry. Not even the ZFS devs can rewrite block pointers on a running filesystem.[1]) That being said ZFS has an awesome feature called delegation which lets you run a subset of commands without root permissions. I use it sparingly, mostly to allow unprivileged backups. (Each host gets a login on my backup server, and can only create/destroy datasets that are children of a host-specific dataset. So `nagato` can
this video The above video is an explanation of a bunch of the barriers to implementing block pointer rewrites. The conclusion is that it would make the code a lot more complicated and break a lot of the layering, and probably make addition of other new features a lot harder. Even a standalone offline rewriting tool wouldn't necessarily be accepted into the OpenZFS codebase because of the maintenance burden. Their advice is that if you think you need that feature to solve a particular problem, you should be looking for a workaround to solve that problem without requiring the huge block pointer rewrite project (which nobody's working on), even if the workarounds have a significant and permanent performance impact. When you take into account how long the feature's been in demand and been on the roadmaps under "eventually", it's clearly not going to happen anytime soon and won't happen without a major change to how ZFS development is being done. It's not definitely impossible, but it's perpetually several years away from happening. With btrfs already having it's equivalent to that feature and stealing an ever-growing slice of the users who need that feature, it's probably never going to happen for ZFS.
No, there is no defragmentation for ZFS, unfortunately. A way to get around that is to send the pool's content to another (fresh) ZFS pool, where it would be written sequentially. But for that you would need a set of drives of same (or larger) capacity. There are ideas on how one would do an actual defrag. They are generally based on a concept called block pointer rewrite, which Matt Ahrens once said could be the 'last feature ever implemented in ZFS', as it would make everything so much more complicated, that it would be hard to add new features afterwards [1]. [1] this video (Link to the beginning of the explanation, the 'last feature ever implemented' quote is at at around 50:25)
> Illumos has a way to expand pools ... ZFS creates a vdev inside the new larger vdev Huh? > IDK if that's in OpenZFS yet. The openzfs tree (on github) is virtually identical to illumos-gate (on github). > physical block addresses should never have been part of the ZFS Merkle hash tree, thus allowing physical addresses to change without having to rewrite every block from the root down. mahrens deals with this (and block pointer rewriting) here: this video Even with SSDs IOPS are precious. On rotating media, burning track-to-track seeks in reading and updating a large hash table is a bad plan (cf. the deduplication table).