Graham Christensen

One should make a distinction between:

* The general idea of mixing together filesystems+folders to achieve re-use/sharing/caching.

* The "Dockerfile" approach to this - with its linear sequence of build-steps that map to a linear set of overlays (where each overlay depends on its predecessor).

The "Dockerfile" approach is pretty brilliant in a few ways. It's very learnable. You don't need to understand much in order to get some value. It's compatible many different distribution systems (apt-get, yum, npm, et al).

But although it's _compatible_ with many, I wouldn't say it's _particularly good_ for any one. Think of each distribution-system -- they all have a native cache mechanism and distribution infrastructure. For all of them, Dockerization makes the cache-efficacy worse. For decent caching, you have to apply some adhoc adaptations/compromises. (Your image-distribution infra also winds up as a duplicate of the underlying pkg-distribution infra.)

Here's an alternative that should do a better job of re-use/sharing/caching. It integrates the image-builder with the package-manager:

https://grahamc.com/blog/nix-and-layered-docker-images/

Of course, it trades-away the genericness of a "Dockerfile', and it no doubt required a lot of work to write. But if you compare it to the default behavior or to adhoc adaptations, this one should provide better cache-efficacy.

(All this is from POV of someone doing continuous-integration. If you're a downstream user who fetches 1-4…

These are all good points. To be honest, I'm skeptical about the future of NixOS: it's wonderful as a user distribution, but for production servers it's a hard sell to migrate off a stable, enterprise-grade OS like CentOS, Debian, or openSUSE. And as you say there's a long-term trend toward cattle servers and lightweight throwaway containers, which do not need features like atomic rollback and deterministic versioning, because we can simply snapshot the image and do a blue-green deployment with easy rollback at the orchestration layer.

That said, NixPkg (the package manager) is incredibly powerful, because it can build containers more quickly than Dockerfiles [1] or other specifications -- and it can do so in an entirely reproducible manner, with efficient caches for CI servers and developers. This becomes particularly powerful if your team works in a compiled language with long build times, like C++, or if your team works with a variety of languages. Particularly in scientific computing, it's not uncommon to have a lot of Python {2,3} with C/C++ dependencies, and perhaps a cmdline utility in another language or two. At this point something like 'pip install' no longer really cuts it, and NixPkg can step in to produce app images and development/CI environments in a snap. These images can then be backed into your favorite container runtime with the confidence that you could rebuild the entire container with a 1-line patch and actually have the same contents everywhere, and…

>"grahamc’s “Erase your darlings” blog post is an amazing example of what a

snapshotting filesystems (zfs) combined with an immutable, infrastructue-as-code OS (NixOS) can achieve.

To summarize the post, grahamc demonstrates how to erase the root partition at boot while opting in to state by getting NixOS to symlink stuff to a dedicated partition.

This restores the machine to a clean state on every boot, preserving the “new computer smell”.

I believe the main selling point of this concept of opt-in state is that it makes it dead simple to keep track of ephemeral machine state (everything not explicitly specified by your NixOS configuration) and enforces elimination of Configuration Drift. While the benefits of this are clear for servers, this also works pretty well with workstations and laptops, where you gradually accumulate junk in /etc and /var which you never can be completely confident in deleting.1"

Related:

https://dzone.com/articles/configuration-drift

https://grahamc.com/blog/erase-your-darlings

https://news.ycombinator.com/item?id=22856199

There are some technical objections to the design of flakes, but the controversy in the community is primarily social imo.

The reading I'd recommend as someone sympathetic to both 'sides' isn't really a technical comparison, but this: https://grahamc.com/blog/flakes-are-an-obviously-good-thing

As far as technical pros and cons, it's worth looking at colemickens' PoC demonstrating that in principle, you can use Nix to generate literally identical output with and without flakes: https://github.com/colemickens/nixos-flake-example

> Both approaches

it's also worth noting that prior to flakes, there was no single alternative approach to the main problems flakes aims to solve. There were (and are) a whole bunch of competing and bespoke practices for pinning nixpkgs and other Nix sources. You can take a look at niv and nix-thunk to get a sense of them if you want

I personally remember being inspired by Erase your Darlings and Paranoid NixOS Setup back in the day, less for the hardening measures and more because of how great the Nix syntax looked. Huge, monumental ass-pain setups could be scripted away in one or two lines like it was nothing. You could create wildly optimized configurations for your specific use-case, and then divide them into modules so they're portable.

It's not advisable to switch to one of these paranoid configurations outright, but they're a great introduction to the flexibility provided by the NixOS configuration system. I'd also recommend Xe's documentation of Nix Flakes, which can be used on any UNIX-like system including macOS: https://xeiaso.net/blog/nix-flakes-1-2022-02-21/

https://grahamc.com/blog/erase-your-darlings/

https://xeiaso.net/blog/paranoid-nixos-2021-07-18/

I use colmena[1] to manage one nixos configuration for multiple machines:
- laptop
- desktop
- server
- rpi nas

I also wipe my entire rootfs every boot with a zfs snapshot rollback[2] using the impermanence module[3] to keep specific stateful data one one of two datasets with regular snapshots: one is backed up with zfs send, the other is just for cache between reboots.

It took a little puzzling to get started, because I didn’t know about the impermanence module at first, so I built my own hacky solution. But I really love this setup. And the way I don’t have cruft to clean.

Also my backups are so much smaller now :’-)

[1]: https://colmena.cli.rs/

[2]: https://grahamc.com/blog/erase-your-darlings/

[3]: https://github.com/nix-community/impermanence