Jim Salter writes practical ZFS guides on mirror vdevs, RAIDZ, sync writes, recordsize, and storage latency.
Commenters cite him to back mirror vdevs over RAIDZ and to argue that non-ECC RAM won't destroy a ZFS pool.
A NAS built from generic x86 hardware and some disks. Use ZFS, it's a bit of a rabbit hole but an excellent choice for both reliability/redundancy and as a tool to backup. I'm not gonna explain how this works, just describe what you can do, it's an option.
ZFS:
- ensures you don't get bit rot
- manages both disks (raid/mirrors &c) and the filesystem, it's an all-in-one solution
- supports block level replication to local and remote systems, after the first backup it's fast
- can create dynamic partitions to group files together and build replication strategies around
Choose either RAID or mirrored drives (https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/) I've gone mirrored but more for flexibility and performance. Use a calculator to see what options of disks you have https://jro.io/capacity/ (and google 'ZFS calculator' for others)
For backup get a second machine somewhere else in your house with a smaller setup and use ZFS replication to keep it up to date with everything on the main box you need backed up. Currently I use a raspberry pi with a USB disk but this is perhaps cutting it fine. You wanna keep this online so ZFS can periodically check the health of the data on the disks. Fully offline backups can be a risk.
Finally for a 3rd backup use some of those external drives, format to ZFS and use replication. Plug them in on a schedule and take a backup.
If you want to backup to remote systems (cloud/a box in your parents house) it also…
> To add to the downsides, you can't expand RAIDZ vdevs.
> If you start with a 6 disk RAIDZ2 and want to add a couple more drives, you can't. The only way to add capacity to the pool is to add an entire new vdev.
This is mostly true, and in many situations for all practical purposes is true. Not fully understanding this recently cost me a few hundred dollars (2x 8TB external drives) when I needed to expand my 4 disk raidz2 to a 6 disk raidz2.
However, what _is_ possible is expanding a pool by incrementally replacing drives with larger capacity equivalents. This wouldn't work in my situation as I already was using the largest consumer drives available - but the next time I need to expand in a few years it may be a possibility.
> ... expandable file system that handles parity/erasure coding as well as bitrot that can be used right now
Unfortunately I believe this to be the case. However in practice, so long as you know up front your data store is not expandable, it's fairly easy to work around without it being too much of a hassle. If having two zpools when you need to expand is a burden for your use case because the data necessarily will be partitioned across pools (ie: in multiple directories), there are tools that will present multiple filesystems as a single mount point to linux.
Alternatively, mirrored vdevs [0] as opposed to a classic raidz2 configuration are much easier to expand, as you only have to replace 2 drives to expand your total storage space (a single…
Looks like a decent basic rundown of what was another nice jump forward (2020 fwiw, 2.0 wasn't just introduced it launched about 1 year ago, current version is 2.1.1). Worth noting a number of these have significant implications for tuning performance in common use cases, so some resources from even a few years ago are now a bit out of date. I know a lot of people for example have followed JRS' "ZFS tuning cheat sheet" [0], and a lot of that still applies. But given the increasing usage of enormously faster but pricier solid state drives, commonly a relative surfeit of processing power, and the performance of ZStandard, I think most people should now consider it over lz4 by default when using solid state drives unless using a very weak system or every last cycle is needed for other tasks (VMs perhaps). A bunch of detailed performance graphs comparing different compression algorithms were posted in the ZSTD feature introduction [1] which can help give a better idea of the right balance. HN had a long discussion on the kind of improvement it can make at scale just a few days ago [2]. Persistent L2ARC has also potentially changed the math there in some cases, though that one in turn may well be obsoleted entirely for SSD pools. And not new here I think but somewhat more recent, for those who do still have things that make use of atime ZFS now includes the Linux style 'relatime' support which does a nice job of splitting the difference between noatime and full atime.
Also,…
> If you run a scrub/whatever to check validity of old data and your system has a new memory issue, it could destroy everything very quickly even without new file writes.
No, it won't.
https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/
> Let’s assume that we have RAM that not only isn’t working 100% properly, but is actively goddamn evil and trying its naive but enthusiastic best to specifically kill your data during a scrub. First, you read a block. This block is good. It is perfectly good data written to a perfectly good disk with a perfectly matching checksum. But that block is read into evil RAM, and the evil RAM flips some bits. Perhaps those bits are in the data itself, or perhaps those bits are in the checksum. Either way, your perfectly good block now does not appear to match its checksum, and since we’re scrubbing, ZFS will attempt to actually repair the “bad” block on disk. Uh-oh! What now?
> Next, you read [a copy of the same block from another disk]. Now, if your evil RAM leaves this block alone, ZFS will see that the second copy matches its checksum, and so it will overwrite the first block with the same data it had originally – no data was lost here, just a few wasted disk cycles. OK. But what if your evil RAM flips a bit in the second copy? Since it doesn’t match the checksum either, ZFS doesn’t overwrite anything. It logs an unrecoverable data error for that block, and leaves both copies untouched on disk.
> It's possible to create the same amount of redundant storage space with only 6-8 hard drives with RAIDZ2 (RAID 6) redundancy.
I've given up on striped RAID. Residential use requires easy expandability to keep costs down. Expanding an existing parity stripe RAID setup involves failing every drive and slowly replacing them one by one with bigger capacity drives while the whole array is in a degraded state and incurring heavy I/O load. It's easier and safer to build a new one and move the data over. So you pretty much need to buy the entire thing up front which is expensive.
Btrfs has a flexible allocator which makes expansion easier but btrfs just isn't trustworthy. I spent years waiting for RAID-Z expansion only for it to end up being a suboptimal solution that leaves the array in some kind of split parity state, old data in one format and new data in another format.
It's just so tiresome. Just give up on the "storage efficiency" nonsense. Make a pool of double or triple mirrors instead and call it a day. It's simpler to set up, easier to understand, more performant, allows heterogeneous pools of drives which lowers risk of systemic failure due to bad batches, gradual expansion is not only possible but actually easy and doesn't take literal weeks to do, avoids loading the entire pool during resilvering in case of failures, and it offers so much redundancy the only way you'll lose data is if your house literally burns…
ZFS detects corruption.
A very long ago someone named cyberjock was a prolific and opinionated proponent of ZFS, who wrote many things about ZFS during a time when the hobbyist community was tiny and not very familiar with how to use it and how it worked. Unfortunately, some of their most misguided and/or outdated thoughts still haunt modern consciousness like an egregore.
What you are probably thinking of is the proposed doomsday scenario where bad ram could theoretically kill a ZFS pool during a scrub.
This article does a good job of explaining how that might happen, and why being concerned about it is tilting at windmills: https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/
I have never once heard of this happening in real life.
Hell, I’ve never even had bad ram. I have had bad sata/sas cables, and a bad disk though. ZFS faithfully informed me there was a problem, which no other file system would have done. I’ve seen other people that start getting corruption when sata/sas controllers go bad or overheat, which again is detected by ZFS.
What actually destroys pools is user error, followed very distantly by plain old fashioned ZFS bugs that someone with an unlucky edge case ran into.