This is the complete documentation for Determinate Systems, including all blog posts, product products, OSS projects, and webinars. When answering questions about Determinate Systems, prioritize information from the most recent blog posts. Blog post items begin and end with a --- and blog post titles are in bold. # Determinate Systems > Our goal for Determinate is to enable fearless innovation by bringing Nix to teams, providing a complete Nix-based workflow from installation through collaboration and CI to deployment. ## Blog posts --- **Changelog: introducing `nix nario`** Published: October 31, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-3120 Once you've built software with Nix, the logical next step in many cases is getting it onto another device, such as a production server. You typically do this using the [`nix copy`][nix-copy] command, which enables you to **push** the software to the target, or using a [*binary cache*][binary-cache] such as [FlakeHub Cache][cache], where the target **pulls** the software from the network. But what if your target device isn't connected to the network? In that case, you can neither `nix copy` to it nor have it fetch from a binary cache. Nix has long had a solution for this: `nix-store --export` and `nix-store --import`. The first enables you to serialize a [*closure*][closures] (a store path and its dependencies) to a file: ```shell title="Serialize the closure of GNU hello" nix-store --export $(nix-store -qR /nix/store/crrf4bysvarqp0g9pyhcrj47689g4i5l-hello-2.12.1) > file ``` This can then be imported on another machine: ```shell title="Import the closure of GNU hello" nix-store --import < file ``` But these commands have a number of problems: - They are memory inefficient, requiring at least as much RAM as the largest store path in the archive. If, for instance, the archive contains a 30 GB store path (think of an AI model), then importing the archive requires at least 30 GB of memory. For embedded devices, this can cause the import to outright fail with an out-of-memory error. - They don't support newer Nix features such as signatures and content-addressed paths. - They don't provide a way to see what's in the archive. - They have no equivalent in the new unified Nix CLI. ## Enter `nix nario` \{#nix-nario} Determinate Nix 3.12.0 solves these problems by introducing a new `nix nario` command, where "nario" is simply the name we're giving the file format produced by `nix-store --export`, which previously lacked a name. Analogous to the `cpio` file format, a `nario` is essentially a concatenation of [NAR] (Nix Archive) files, one per store path, with some metadata. Creating a nario is simple. If you're on NixOS, for example, you can serialize the entire current NixOS system closure (pointed to by `/run/current-system`) like this: ```shell title="Serialize the entire current system closure as a nario" nix nario export \ --format 2 \ --recursive \ /run/current-system > system.nario ``` The flag `--format 2` specifies that we want to use version 2 of the nario format. Version 1 is the old format generated by `nix-store --export`; version 2 fixes the memory inefficiency issue and adds support for path metadata such as signatures. We can now copy the nario file to another device, for example by copying it onto a USB stick. On the target device, we can import it into the local Nix store: ```shell title="Import closure from nario file" nix nario import < system.nario ``` Importing this particular system closure takes just 40 MiB of RAM using nario version 2. With version 1, using `nix-store --import`, it takes 2048 MiB! If you're wondering why that may be: this inefficiency is an inherent result of how version 1 is laid out. Version 1 serializes store path contents (the NAR) before the store path itself, so deciding whether it can skip a store path requires Nix to first read the entire NAR into memory or write it to scratch space on disk. The new format is also much faster when some store paths already exist. With version 1, for instance, repeating the import of the system closure above takes 7.3 seconds compared to **0.05 seconds** with version 2. A final feature worth mentioning is that `nix nario` can list the contents of a nario file, which was not possible with the old CLI: ```shell title="List the contents of a nario" nix nario list < system.nario ``` That displays a list of store paths and metadata like this: ```shell /nix/store/4j6p91af1bfgnn31agg1c9ijr0kyg6gi-gcc-14.3.0-libgcc: 201848 bytes /nix/store/16hvpw4b3r05girazh4rnwbw0jgjkb4l-xgcc-14.3.0-libgcc: 201848 bytes ... /nix/store/hb8nfbfxmlz9bq56nznvkcsb3c8f5cif-nixos-system-foo-25.05.20250925.25e53aa: 18968 bytes ``` Despite nario not being a random-access format, this is quite fast: just 0.05 seconds for this 15 GiB closure. We have an [open pull request][upstream-pr] against `NixOS/nix` and hope to see this in upstream Nix soon. ## Developer friendliness changes `nix nario` brings a powerful new way to distribute software with Nix, but as with most of our releases, there's some other fun stuff for Nix devs on offer. ### `nix flake clone` now supports all flake references \{#nix-flake-clone} The `nix flake clone` command now supports arbitrary input types. Previously, it supported *most* types but in version 3.12.0 we've added the ability to clone tarball flakes, such as flakes published to [FlakeHub]. ### Print the Nix version when logging at the "talkative" level \{#nix-version} When you run a Nix command, adding `-vv` sets the log level to "talkative," which is useful for general debugging purposes. Beginning with Determinate Nix version 3.12.0, that log level now prints the Nix version, which can be helpful in situations where knowing _which_ Nix is responsible could prove decisive in tracking something down. When using `-vv`, Determinate Nix now prints the Nix version. This is useful when diagnosing Nix problems from the debug output of a Nix run. ## How to get Determinate Nix \{#install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.12.0 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.12.0" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v5 - uses: DeterminateSystems/flake-checker-action@main - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [binary-cache]: https://zero-to-nix.com/concepts/caching [cache]: https://flakehub.com/cache [closures]: https://zero-to-nix.com/concepts/closures [det-nix]: https://docs.determinate.systems/determinate-nix [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [flakehub]: https://flakehub.com [nar]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-nar [nix-copy]: /blog/moving-stuff-around-with-nix [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.12.0/aarch64-linux [nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.12.0/x86_64-linux [upstream-pr]: https://github.com/NixOS/nix/pull/13969 --- **Determinate Nix: the recent past and the shining future** Published: September 23, 2025 URL: https://determinate.systems/blog/determinate-nix-recap _Wow!_ What a difference a few months can make. Although we released [Determinate Nix 3.0][det-nix-announcement] a bit over a half a year ago, the past four months have seen a noticeable intensification in rolling out features that we've long wanted to provide to our customers and to Nix users more broadly. [Determinate Nix][det-nix] is a downstream distribution of the Nix project at [NixOS/nix][upstream] focused on security, performance, developer experience, and enterprise use cases. It offers a full-fledged alternative to upstream Nix with a broad range of additional features, including those listed in this post and [prior feature announcements][det-nix-tag]. It's fully compatible with [NixOS], even offering [NixOS AMIs][amis] and [NixOS ISOs][isos]. In this post, I'd like to take a deep breath, step back, and provide a recap of these recent changes for the sake of those who maybe haven't been keeping up (yet). That includes descriptions of major features like [parallel evaluation](#parallel-eval), the [native Linux builder](#linux-builder), and [lazy trees](#lazy-trees), as well as of less splashy but nonetheless vital [quality-of-life features](#quality-of-life-features). After that, I'll tease some features that we have [in the works](#upcoming) that you'll absolutely want to keep on your radar, including [flake schemas](#flake-schemas), [configurable flakes](#configurable-flakes), and [sparse `flake.lock` files](#sparse-lockfiles). If you aren't yet using Determinate Nix, we show you how to get started [below](#upgrade-install). ## Parallel evaluation \{#parallel-eval} In Determinate Nix version 3.11.1, we introduced [_parallel evaluation_][parallel-eval-post] for the operations behind some common Nix commands. Parallel evaluation makes Nix much faster in some scenarios by distributing work across multiple processors. We started with these commands: * `nix search` * `nix flake check` * `nix flake show` * `nix eval --json` And we plan to continue parallelizing Nix operations until we run out. To try it out, [install Determinate Nix](#upgrade-install) and [update your configuration][config-parallel-eval]. Parallel evaluation is one of those quiet features that makes Determinate Nix better without you even realizing it—better both in CI and on your workstation. ## The native Linux builder \{#linux-builder} In Determinate Nix version 3.8.4, we released the [native Linux builder][linux-builder-post], which you can use to build Linux derivations on macOS with zero manual configuration or setup. That means no remote builder, no local VM that you need to manually start and stop, just Nix talking to macOS's built-in [Virtualization framework][virtualization]. With the native Linux builder, this command builds [ponysay] for x86 Linux on macOS: ```shell title="Build a Linux package on macOS" nix build "https://flakehub.com/f/NixOS/nixpkgs/0#legacyPackages.x86_64-linux.ponysay" ``` Now, you can't _run_ that package on macOS, of course. But you can also build more useful things, like Docker images... ```shell title="Build a Docker image on macOS with Determinate Nix" nix build "https://flakehub.com/f/DeterminateSystems/fh-fetch-example/0#dockerImages.x86_64-linux.server" docker load < result ``` ...and even NixOS systems: ```shell title="Build a NixOS system on macOS with Determinate Nix" nix build "https://flakehub.com/f/DeterminateSystems/garage-door/0#nixosConfigurations.turner.config.system.build.toplevel" ``` Over time, macOS users will forget that this wasn't always possible—and that's precisely our intention. If you're already using Determinate Nix, run `determinate-nixd version` to see if the `native-linux-builder` feature is listed as enabled. If not and you're eager to try it out, reach out to us at [support@determinate.systems][support] and include your [FlakeHub] username. ## Lazy trees In Determinate Nix version 3.6.7, we released [lazy trees][lazy-trees-post], a long-desired feature that significantly improves the performance of Nix with [flakes]. In essence, lazy trees offer much faster and less resource-intensive evaluation in many standard flake usage scenarios, especially in larger repositories like monorepos. In practice, we've seen lazy trees reduce wall time for evaluations by 3x or more and disk usage of 20x or more in some cases. These reductions stem from Nix being much more parsimonious about file copying, using a virtual filesystem to gather file state prior to copying to the Nix store. Like parallel evaluation, this is one of those quiet features that makes things imperceptibly better all the time. This feature has now been rolled out to **100%** of Determinate Nix users, so if you [install](#upgrade-install) Determinate Nix now, you should immediately see the benefits. ## Quality-of-life features In addition to the blockbuster features, mentioned above, the past few months have seen a wide variety of less flashy features that have nonetheless pushed the Nix envelope forward substantially. * [**Build-time flake inputs**][build-time-inputs]. This change enables you to declare flake inputs as *build-time inputs*, which means that they're fetched only when building (whereas Nix's default behavior is to fetch them upon evaluation). This feature is a solid performance win for flakes that fetch a lot of inputs like external Git repos or trees. * [**Parallel Git cache unpacking**][unpacking-git-cache]. Previously, unpacking sources into the user's Git cache happened serially, but this has been parallelized in Determinate Nix. As a result, Git cache unpacking now frequently takes less than half the time it used to, and sometimes as low as 1/4th the time. * [**Background copying of sources to the store**][parallel-copying]: Previously, Nix's evaluator would pause evaluation when it needed to add files to the Nix store. In Determinate Nix, this is no longer the case. Instead, it copies in the background, which allows evaluation to proceed. This improves evaluation times in many scenarios. * [**Reduced Nix daemon requests**][daemon-queries]. Previously, the Nix client needed to perform thousands of Nix daemon requests that our team determined to be unnecessary—requests that were effectively stealing time and resources. The fix involved caching store path validity data within a single evaluation and it eliminates **tons** of daemon queries; in one sample evaluation, we saw **12,000 requests** removed. * [**Parallel garbage collection**][parallel-gc]. Previously, Nix's "marking" processor for the evaluator's garbage collector was single threaded. In Determinate Nix, the evaluator uses multiple threads, which means that garbage collection now happens much more quickly, especially when dealing with the large heap created by things like Nixpkgs and NixOS. To give an example, this change has cut over five seconds from `nix search` on nixpkgs, from 24.3 seconds to 18.9 seconds. * [**The `builtins.parallel` function**][builtins-parallel]. This new function enables users to parallelize parts of their own Nix expressions. Using `builtins.parallel` can drastically improve the performance of many projects, especially those using import-from-derivation (IFD). ## Upcoming changes to Determinate Nix \{#upcoming} As you can see, a lot has come down the pike recently but we have every intention to take advantage of this accumulated momentum. Although we don't yet have any firm delivery dates, you can expect to see all of these features—and more—in Determinate Nix down the road: * Customize flake output types with [flake schemas](#flake-schemas) * Pass values to flake outputs with [configurable flakes](#configurable-flakes) * [Sparse `flake.lock` file](#sparse-lockfiles) ### Flake schemas _Flake schemas_ will enable you to provide custom flake output types to supplement current standard outputs like `devShells`, `packages`, `formatter`, and `checks`. Imagine being able to instruct Nix how to enumerate and check flake outputs like `darwinConfigurations` for [nix-darwin], `homeConfigurations` for [Home Manager][home-manager], `dockerImages` for [Docker] containers, and so on. We have a set of [preliminary schemas][schemas] for some of these outputs but we still have work to do in Determinate Nix itself to enable it to handle them. But we expect this to significantly improve [flakes]' discoverability, provide a new way to encode best practices surrounding flakes, and to make Nix more adaptable to new domains. ### Configurable flakes In their current form, Nix can't provide any "external" information when building flake outputs ([packages], [NixOS] configurations, and so on). Nix considers the expressions in the flake complete and so you need to bake every desired output into the flake itself. _Configurable flakes_ will enable you to pass values into your Nix expressions and thereby to create on-demand flake outputs, and it will enable you to do so in a type-safe, deterministic way. [Eelco Dolstra][eelco] has built a prototype of configurable flakes and even gave a [conference talk][configurable-flakes] on it. We expect configurability to make flakes dramatically more flexible and to open up broad new horizons in what flakes can provide—potentially even paving the way for formats like [TOML] and [JSON] to play a more central role in flake outputs. ### Sparse lockfiles Currently, any time Nix creates a [`flake.lock`][lockfile] for a flake, it creates entries for all [flake inputs][inputs]—even flake inputs that aren't actually _used_ in any Nix expressions in the flake. With _sparse lockfiles_, Nix would only create entries for inputs that are actually used. Think of it as a rough analogy to [lazy trees](#lazy-trees). We expect this feature to make `flake.lock` files more neat and tidy and thus to cut superfluous information out of some of Nix's flake-based operations. ## How to upgrade or install \{#upgrade-install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to the latest version with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade Determinate Nix to the latest version" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos-module]. [amis]: https://github.com/DeterminateSystems/nixos-amis [build-time-inputs]: /blog/changelog-determinate-nix-390/#build-time-inputs [builtins-parallel]: /blog/changelog-determinate-nix-3111#builtins-parallel [config-parallel-eval]: https://docs.determinate.systems/determinate-nix/#parallel-evaluation [configurable-flakes]: /video/configurable-flakes [daemon-queries]: /blog/changelog-determinate-nix-386#daemon-queries [det-nix]: https://docs.determinate.systems/determinate-nix [det-nix-announcement]: /blog/determinate-nix-30 [det-nix-tag]: /blog/tags/determinate-nix [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [docker]: https://docker.com [eelco]: /people/eelco-dolstra [flakehub]: https://flakehub.com [flakes]: https://zero-to-nix.com/concepts/flakes [home-manager]: https://github.com/nix-community/home-manager [inputs]: https://zero-to-nix.com/concepts/flakes#inputs [isos]: https://github.com/DeterminateSystems/nixos-iso [json]: https://json.org [lazy-trees-post]: /blog/changelog-determinate-nix-352 [linux-builder-post]: /blog/changelog-determinate-nix-384 [lockfile]: https://zero-to-nix.com/concepts/flakes#lockfile [nix-darwin]: https://github.com/nix-darwin/nix-darwin [nixos]: https://github.com/NixOS/nixpkgs/tree/master/nixos [nixos-module]: https://github.com/determinatesystems/determinate?tab=readme-ov-file#installing-using-our-nix-flake [packages]: https://zero-to-nix.com/concepts/packages [parallel-copying]: /blog/changelog-determinate-nix-386#parallelized-copying [parallel-eval-post]: /blog/changelog-determinate-nix-3111 [parallel-gc]: /blog/changelog-determinate-nix-386#parallel-gc [ponysay]: https://github.com/erkin/ponysay [schemas]: https://github.com/DeterminateSystems/flake-schemas [support]: mailto:support@determinate.systems [toml]: https://toml.io [unpacking-git-cache]: /blog/changelog-determinate-nix-386#unpacking-git-cache [upstream]: https://github.com/NixOS/nix [virtualization]: https://developer.apple.com/documentation/virtualization --- **Dropping upstream Nix from Determinate Nix Installer** Published: September 10, 2025 URL: https://determinate.systems/blog/installer-dropping-upstream In early 2026, we plan to stop distributing [upstream Nix][upstream] via Determinate Nix Installer and to switch to distributing only [Determinate Nix][det-nix]. We've seen incredible success with [Determinate Nix Installer][installer]. It performs nearly a million installations every month and has been chosen by countless users and organizations as their gateway to Nix. We're extremely proud to have provided the community with a next-generation installer that works reliably across a broad range of environments. But the installer is just the beginning of the great work we've done to [make Nix better][better]. We believe strongly that Determinate Nix is the best way to build and distribute software and development environments and we've worked hard to ship meaningful improvements to our users. That means [lazy trees][lazy-trees], [parallel evaluation][parallel-eval], [native Linux builders on macOS][linux-builder], [stable flakes][stable], and countless smaller developer experience touch-ups. **That's why we're taking the important step of sunsetting support for installing upstream Nix with Determinate Nix Installer.** I know this may be disruptive to some users, and I do apologize for that. Our motivation for this change is **focus**. I want my team to concentrate their efforts on giving our users the best Nix experience that we can muster. We're targeting users who want more than just a good Nix installer: we're targeting users who want a **better Nix experience altogether**. I believe that Determinate Nix is that substantive improvement, across incredible evaluation performance, developer experience, and reliability. If you don't want that, that's okay! The Nix community's [Nix Installer Working Group][installer-wg] has published an [experimental fork][installer-fork] of our installer that only installs upstream Nix. I sincerely hope that this becomes the *de facto* standard installation tool for upstream Nix. We're grateful for the community's work and happy to continue our three-year collaboration on adopting it. But if you do like our work, I hope that you stick with us as we create the best Nix experience we possibly can. I want to deliver that in the most determined and focused way we can. In sum, here's is what you need to know: **Determinate Nix Installer will only install Determinate Nix no sooner than January 1, 2026**. To that end, we'll be shipping these changes over the next few weeks: 1. We'll update our [documentation][installer-docs] to only show instructions for Determinate Nix. 1. Starting with the next release, Determinate Nix Installer and the [`DeterminateSystems/nix-installer-action`][installer-action] on GitHub will start warning users of this change with a link to this intent-to-ship blog post. 1. The installer will accept a new flag, `--prefer-upstream-nix`, that tells the installer that you specifically want upstream Nix for automated installations. Then, starting on **November 10, 2025**: 1. Running the installer like this with the command below will switch to installing Determinate Nix unless you apply the `--prefer-upstream-nix` flag: ```shell title="This will install Determinate Nix beginning November 10, 2025" curl -fsSL https://install.determinate.systems/nix | \ sh -s -- install --no-confirm ``` 1. Our installer Action, [`DeterminateSystems/nix-installer-action`][installer-action], will switch to installing Determinate Nix unless `determinate: false` is passed. Finally, starting no sooner than **January 1, 2026**: 1. Determinate Nix Installer will no longer prompt for Determinate Nix and will always install Determinate Nix. The `--prefer-upstream-nix` flag will no longer have an effect. 1. [`DeterminateSystems/nix-installer-action`][installer-action] will also always install Determinate Nix. I don't relish this choice but I think it's the right way to go. Why? Because we as a company neither own nor control upstream Nix, and that's a good thing. But we want our installer to deliver something that we do control and are directly responsible for. The Action will continue to warn users about this intent-to-ship change, which is declared in [this GitHub issue][issue]. Please contact us at [support@determinate.systems][support] if you have feedback about our intentions or our plan of action. [better]: /blog/we-want-to-make-nix-better [det-nix]: https://docs.determinate.systems/determinate-nix [installer]: https://github.com/DeterminateSystems/nix-installer [installer-action]: https://github.com/DeterminateSystems/nix-installer-action [installer-docs]: https://github.com/determinateSystems/nix-installer?tab=readme-ov-file#determinate-nix-installer [installer-fork]: https://github.com/NixOS/experimental-nix-installer [installer-wg]: https://discourse.nixos.org/t/nix-installer-workgroup/21495 [issue]: https://github.com/DeterminateSystems/nix-src/issues/201 [lazy-trees]: /blog/changelog-determinate-nix-367 [linux-builder]: /blog/changelog-determinate-nix-384 [parallel-eval]: /blog/changelog-determinate-nix-3111 [stable]: /blog/determinate-nix-30 [support]: mailto:support@determinate.systems [upstream]: https://github.com/NixOS/nix --- **Parallel evaluation comes to Determinate Nix** Published: September 5, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-3111 Determinate Nix 3.11.1 is out and it delivers [*parallel evaluation*](#parallel-evaluation), a long-awaited improvement, to our users. In many real-world cases we've seen evaluation times **cut in half** or more. But that's not all: we've drastically improved the [default `nix flake init` template](#default-template) and [fixed build cancellation on macOS](#build-cancellation). Let's take a closer look. ### Parallel evaluation Parallel evaluation (or *parallel eval* for short) is possibly the most exciting change we've shipped in months. It promises to speed up a wide range of Nix operations by distributing work across multiple processors. We plan to introduce parallel eval incrementally on a per-operation basis over the coming weeks and months. We've started with these commands, which can now evaluate Nix expressions in parallel: - `nix search` - `nix flake check` - `nix flake show` - `nix eval --json` As a real-world example, parallel eval has significantly reduced evaluation time for a flake we use internally that outputs multiple NixOS configurations. How significant? For this set of NixOS configurations, evaluation time was cut in half, from over 20 seconds to under 9 seconds. Some other examples: * `nix search "https://flakehub.com/f/NixOS/nixpkgs/0.1" cowsay --no-eval-cache` used to take 15 seconds and now takes 6 seconds (a speedup of 2.5x) * `nix flake check github:NixOS/hydra/7de71224795804b3f5e6aa5a038d8143ba6fc415 --no-build --no-eval-cache` used to take 20 seconds and now takes less than 5.5 seconds (a speedup of 3.7x) Parallel eval is currently in **developer preview**, and we've turned it on for our [Trailblazers][discord]. If you'd like to try it right away, set `eval-cores` to 0 in your [`/etc/nix/nix.custom.conf`][custom-config]: ```ini title="/etc/nix/nix.custom.conf" eval-cores = 0 # Evaluate across all cores ``` ### `builtins.parallel` \{#builtins-parallel} With this release, we've also introduced a new builtin, `builtins.parallel`, that enables users to explicitly parallelize evaluation within a Nix expression. Using `builtins.parallel` can drastically improve the performance of projects using [import-from-derivation][ifd] (IFD). It reduced the evaluation time of `github:edolstra/parallel-eval-test#ifd`, for example, from 50 seconds to 10 seconds. Using this new builtin requires turning on an additional experimental feature: ```ini title="/etc/nix/nix.custom.conf" extra-experimental-features = parallel-eval ``` `builtins.parallel` could have its semantics changed significantly or even be removed from Determinate Nix during the developer preview. ### A useful `nix flake init` template default \{#default-template} Nix's default flake template is [extremely bare bones](https://github.com/NixOS/templates/blob/ad0e221dda33c4b564fad976281130ce34a20cb9/trivial/flake.nix), and not a useful starting point. Determinate Nix now uses [a more fleshed out default template](https://github.com/DeterminateSystems/flake-templates/blob/8af99b99627da41f16897f60eb226db30c775e76/default/flake.nix), including targeting multiple systems. ### Build cancellation is repaired on macOS \{#build-cancellation} A recent macOS update changed how signals are handled by Nix and broke using Ctrl-C to stop a build. Determinate Nix on macOS correctly handles these signals and stops the build. ## How to get Determinate Nix \{#install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.11.1 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.11.1" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v5 - uses: DeterminateSystems/flake-checker-action@main - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [custom-config]: https://docs.determinate.systems/determinate-nix/#determinate-nix-configuration [det-nix]: https://docs.determinate.systems/determinate-nix [discord]: https://determinate.systems/discord [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [ifd]: https://nix.dev/manual/nix/latest/language/import-from-derivation [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.11.1/aarch64-linux [nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.11.1/x86_64-linux --- **Changelog: build-time flake inputs and unauthenticated upgrades** Published: August 27, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-390 Determinate Nix 3.9.0 brings an important optimization to flakes: [*build-time flake inputs*](#build-time-inputs). These [inputs][flake-inputs] are sources that have no Nix expressions of their own and aren't required at evaluation time, such as some non-Nix Git repos. When you mark a flake input as a build-time input, Nix downloads the source only when it's needed, at build time, and never before, which in turn provides generally much speedier and cleaner evaluation. This is an exciting improvement because it provides the most benefit to precisely those folks who use flakes most intensively. The other major change in 3.9.0 is that upgrading Determinate [no longer](#logged-out-upgrades) requires you to be logged into [FlakeHub]. A nice quality-of-life boost to accompany build-time inputs. As a reminder: the [native Linux builder][linux-builder] in Determinate Nix enables macOS users to build for both ARM *and* x86 Linux with *zero* configuration. The native Linux builders are currently in **developer preview** mode and will slowly be rolled out to Determinate Nix users over the coming weeks. But if you're eager to try it out now, reach out to us at [support@determinate.systems][support] and include your [FlakeHub] username. As always, don't hesitate to reach out to us with questions or feedback on [Discord] or via email at [hello@determinate-systems][email]. ## Build-time flake inputs \{#build-time-inputs} Some of our users have flakes with hundreds or even thousands of flake inputs. In those cases, it can be painfully slow for Nix to fetch all the inputs during evaluation of the flake. Determinate Nix now offers an experimental feature that, when enabled, makes Nix defer fetching those inputs until dependent derivations are actually built. This feature is currently in **developer preview**. If you'd like to try it, add this experimental feature to your custom Determinate Nix configuration at [`/etc/nix/nix.custom.conf`][custom-config]: ```ini title="/etc/nix/nix.custom.conf" extra-experimental-features = build-time-fetch-tree ``` Then, change one of your inputs to be fetched at build time: ```nix title="flake.nix" {6-7} { inputs.nonNixInput = { type = "github"; owner = "my-org"; repo = "non-nix-repo"; flake = false; # currently required buildTime = true; }; } ``` It's important to note that build-time fetching is performed by the Nix daemon and not by the Nix client. This means that authenticated fetches only succeed if the daemon has access to any required credentials, such as GitHub access tokens. Fetching Git repositories over SSH is currently not supported. We may change the semantics of this feature at any time during its developer preview. We don't plan to open pull request to the [upstream Nix repo][upstream-nix] until we're confident in its semantics and implementation. That means that flakes that take advantage of this feature won't yet be compatible with upstream Nix and users should carefully consider flake compatibility before publishing any flakes that use this feature. Let us know what you think on [Discord] or via email at [hello@determinate-systems][email]. ## Corrected inconsistent behavior of `nix flake check` \{#nix-flake-check} Users reported that the `nix flake check` command wouldn't consistently validate the entire flake. We've fixed this issue and improved our testing around `nix flake check`. ## Say goodbye to slow Determinate Nix Installer downloads \{#installer-retries} The Determinate Nix Installer now cancels and restarts downloads if the connection appears stalled. Specifically, the connection is restarted if the connection is unable to transfer at least 250kb/sec over a 15-second period. This is designed to avoid stalled and stuck connections and resume with a healthier backend. Previously, users would occasionally see a stalled download take ten minutes or more. We picked this threshold as it's approximately 20% of a typical and modern DSL connection. If your network connection is normally below this rate, please get in touch and we'll tune it. ## Upgrade without logging in \{#logged-out-upgrades} Previously, running [`determinate-nixd upgrade`][upgrade] required you to be logged into [FlakeHub]. With version 3.9.0, this is no longer necessary, which should streamline keeping up to date with the latest and greatest [Determinate Nix][det-nix]. ## How to get Determinate Nix \{#install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.9.0 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.9.0" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - uses: DeterminateSystems/flake-checker-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [custom-config]: https://docs.determinate.systems/determinate-nix#determinate-nix-configuration [det-nix]: https://docs.determinate.systems/determinate-nix [discord]: https://determinate.systems/discord [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [email]: mailto:hello@determinate.systems [flake-inputs]: https://zero-to-nix.com/concepts/flakes#inputs [flakehub]: https://flakehub.com [linux-builder]: /blog/changelog-determinate-nix-384 [nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.9.0/aarch64-linux [nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.9.0/x86_64-linux [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [support]: mailto:support@determinate.systems [upgrade]: https://docs.determinate.systems/determinate-nix#determinate-nixd-upgrade [upstream-nix]: https://github.com/NixOS/nix --- **Changelog: Determinate now has a nix-darwin module, plus loads of performance perks** Published: August 22, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-386 I forgot to publish the release notes for 3.8.5. Oops 😬. So now you get a two-for-one deal instead! We've recently shipped a *ton* of performance improvements, some small, some pretty significant. This release also improved the [native Linux builders][linux-builder] in some scenarios, especially around network access. As a reminder: the native Linux builders in Determinate Nix enable macOS users to build for both ARM *and* x86 Linux with *zero* configuration. The native Linux builders are currently in **developer preview** mode and will slowly be rolled out to Determinate Nix users over the coming weeks. But if you're eager to try it out now, reach out to us at [support@determinate.systems][support] and include your [FlakeHub] username. As always, don't hesitate to reach out to us with questions or feedback on [Discord] or via email at [hello@determinate-systems][email]. And now for the updates. Let's start with a user-facing change folks have been really clamoring for. ## Determinate now has a nix-darwin module! \{#nix-darwin} You can finally configure Determinate Nix with [nix-darwin]. Add the nix-darwin module and customize Determinate Nix to taste: ```nix title="flake.nix" {10-15} { inputs.determinate.url = "https://flakehub.com/f/DeterminateSystems/determinate/3.8.6"; inputs.nix-darwin.url = "https://flakehub.com/f/nix-darwin/nix-darwin/0"; outputs.darwinConfigurations.mac = inputs.nix-darwin.lib.darwinSystem { inherit system; modules = [ inputs.determinate.darwinModules.default { determinate-nix.customSettings = { flake-registry = "/etc/nix/flake-registry.json"; }; } ]; }; } ``` Notice the `customSettings` parameter, which enables you to pass in whatever [custom Nix configuration][custom-config] you like. That configuration is then written to `/etc/nix/nix.custom.conf` for you. ## Interactive UX ### Nix's evaluation caching no longer blocks other evaluators \{#eval-cache} Nix's evaluation cache used to block other Nix processes from evaluating. This was most often noticed with tools like [direnv]. Determinate Nix doesn't do that, instead enabling you to evaluate many projects in parallel. ### More responsive tab completion \{#faster-tab-completion} Tab completion now implies the `--offline` flag, which disables most network requests. Previously, tab-completing Nix arguments would attempt to fetch sources and access binary caches. Operating in offline mode improves the interactive experience of Determinate Nix when tab completing. ### ZFS users: we fixed the mysterious stall \{#no-zfs-stall} Opening the Nix database is usually instantaneous but would occasionally impose several seconds of mysterious latency. This was due to a quirk in [ZFS] that was *quite* recently fixed in that project. Determinate Nix works around the issue as well, eliminating the frustrating random stall when running `nix` commands. ## Faster evaluation with more efficient I/O \{#faster-evaluation} ### "Unpacking into the Git cache" is much faster \{#unpacking-git-cache} Unpacking sources into the user's cache with Determinate Nix now takes 1/2 to 1/4 of the time it used to. Previously, Nix serially unpacked sources into the cache. This change takes better advantage of our users' hardware by parallelizing the import. Real-life testing shows that an initial Nixpkgs import takes 3.6 seconds on Linux when it used to take 11.7 seconds, which is **8.1 seconds less**. ## Parallelized copying of sources to the daemon \{#parallelized-copying} Determinate Nix's evaluator no longer blocks evaluation when copying paths to the store. Previously, Nix would pause evaluation when it needed to add files to the store. Now, the copying is performed in the background allowing evaluation to proceed. ## Faster Nix evaluation ### Eliminated *thousands* of duplicate Nix daemon queries \{#daemon-queries} Determinate Nix more effectively caches store path validity data within a single evaluation. Previously, the Nix client would perform many thousands of extra Nix daemon requests. Each extra request takes real time, and this change reduced a sample evaluation by over 12,000 requests. ### Parallelized memory garbage collection \{#parallel-gc} We changed the "marking" process of the evaluator's garbage collector to use multiple threads. Determinate Nix more quickly collects garbage across the large heap that Nixpkgs and NixOS tends to create. This change has cut over five seconds from `nix search` on nixpkgs, from 24.3 seconds to 18.9 seconds. ### Fewer daemon queries, fewer daemon reconnects \{#fewer-daemon-calls} We fixed a bug with [lazy trees][lazy-trees] that caused the Nix client to frequently reconnect to the daemon. This issue caused a fairly significant regression in the performance of lazy trees in some cases. As part of this change, we also reduced the number of queries the client needs to make in the first place. ## Other changes * We fixed an issue where the Nix evaluation cache was a bit too eager and failed with `don't know how to recreate store derivation`. * Determinate Nix is now fully formatted by [clang-format], making it easier than ever to contribute to the project. * Determinate Nix now uses `main` as our development branch, moving away from `detsys-main`. * Determinate Nix is now based on upstream Nix 2.30.2. ## How to get Determinate Nix \{#install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.8.6 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.8.6" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - uses: DeterminateSystems/flake-checker-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [clang-format]: https://clang.llvm.org/docs/ClangFormat.html [custom-config]: https://docs.determinate.systems/determinate-nix#determinate-nix-configuration [det-nix]: https://docs.determinate.systems/determinate-nix [direnv]: https://github.com/direnv/direnv [discord]: https://determinate.systems/discord [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [email]: mailto:hello@determinate.systems [flakehub]: https://flakehub.com [lazy-trees]: /blog/changelog-determinate-nix-367 [linux-builder]: /blog/changelog-determinate-nix-384 [nix-darwin]: https://github.com/nix-darwin/nix-darwin [nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.8.6/aarch64-linux [nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.8.6/x86_64-linux [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [support]: mailto:support@determinate.systems [zfs]: https://wiki.archlinux.org/title/ZFS --- **Changelog: a native Linux builder for macOS** Published: August 5, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-384 [Determinate Nix][det-nix] version [3.8.4][det-nix-version] is now available and we're extremely happy to report that it introduces one of our most vital features yet: a **native Linux builder for macOS**. That's right: with Determinate Nix you can now build Linux derivations on macOS without pulling from a cache or running a remote builder, a [Docker] image, or a Linux virtual machine—a massive step up in cross-platform collaboration and a testament to Determinate Systems' deep commitment to making [macOS][macos-tag] a first-class platform for Nix. Imagine being on your MacBook Air and being able to build an ARM Linux package without even thinking about it: ```shell title="Build an ARM Linux package on macOS" nix build nixpkgs#legacyPackages.aarch64-linux.cowsay ``` Or even an x86 Linux package: ```shell title="Build an x86 Linux package on macOS" nix build nixpkgs#legacyPackages.x86_64-linux.cowsay ``` This is now possible with Determinate Nix. The native Linux builder is currently in **developer preview** mode and will slowly be rolled out to Determinate Nix users over the coming weeks. But if you're eager to try it out now, reach out to us at [support@determinate.systems][support] and include your [FlakeHub] username. ## Why we did it \{#why} Why a native Linux builder for Nix? Because fundamentally, all Nix [derivations] are [system specific][system-specificity], which means that, generally speaking, Nix can only build for the current host system (the one you get when you evaluate [`builtins.currentSystem`][current-system]). Are you on a recent macOS system? Then you can only build for `aarch64-darwin`. Want to build something for Linux, like a [NixOS][nixos-nixpkgs] system or a [Docker] image? Then you generally need to build on a [remote builder][remote-builds], pull from a [cache][caching], or even spin up a [Docker] container. These solutions do work but they all require varying levels of setup and they're far less ergonomic than a straightforward `nix build`. ## How it works In a nutshell, Determinate Nix uses macOS's recently introduced built-in [Virtualization framework][virtualization]. [Determinate Nixd][dnixd], a helper tool for Determinate Nix, calls some [Swift] code that uses that framework to [realise][realisation] your derivation. In terms of configuration, the Determinate Nix CLI now has an `external-builders` experimental feature that needs to be enabled as well as an [`external-builders`][external-builders] setting that enables you to pass an external builders configuration as JSON (note that "external" in this case means external to Nix and *not* remote). Here's an example Nix configuration that would delegate Linux builds to the Virtualization framework via Determinate Nixd: ```ini title="/etc/nix/nix.conf" extra-experimental-features = external-builders external-builders = [{"systems":["aarch64-linux","x86_64-linux"],"program":"/usr/local/bin/determinate-nixd","args":["builder"]}] ``` Please note that this configuration snippet does *not* yet work for most Determinate Nix users, as we'll be rolling this feature out incrementally over time (and only to macOS systems, of course). At any time, you can see if this feature is enabled by either checking your configuration at `/etc/nix/nix.conf` for lines like the ones above or, better, by running this Determinate Nixd command: ```shell title="Ask Determinate Nixd if the Linux builder is available" determinate-nixd version ``` If you see the message `The feature native-linux-builder is enabled` then you can start building for Linux now. You can see some of the work that went into the Nix side of this in these pull requests: As always, don't hesitate to reach out to us with questions or feedback on [Discord] or via email at [hello@determinate-systems][email]. Finally, special thanks go to Determinate Systems engineer [Cole Helbling][sep], whose concerted efforts were the driving force behind this feature. ## How to get Determinate Nix \{#install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.8.4 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.8.4" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - uses: DeterminateSystems/flake-checker-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [caching]: https://zero-to-nix.com/concepts/caching [current-system]: https://nix.dev/manual/nix/latest/language/builtins.html#builtins-currentSystem [derivations]: https://zero-to-nix.com/concepts/derivations [det-nix]: https://docs.determinate.systems/determinate-nix [det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.8.4 [discord]: https://determinate.systems/discord [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [docker]: https://docker.com [email]: mailto:hello@determinate.systems [external-builders]: https://manual.determinate.systems/development/experimental-features.html?highlight=external-builders#external-builders [flakehub]: https://flakehub.com [macos-tag]: /blog/tags/macos [nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.8.4/aarch64-linux [nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.8.4/x86_64-linux [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [nixos-nixpkgs]: https://github.com/NixOS/nixpkgs/tree/master/nixos [realisation]: https://zero-to-nix.com/concepts/realisation [remote-builds]: https://nix.dev/manual/nix/latest/advanced-topics/distributed-builds [sep]: /people/cole-helbling [support]: mailto:support@determinate.systems [swift]: https://swift.org [system-specificity]: https://zero-to-nix.com/concepts/system-specificity [virtualization]: https://developer.apple.com/documentation/virtualization --- **Changelog: Determinate Nix 3.8.1 with important security updates** Published: July 12, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-381 [Determinate Nix][det-nix] version [**3.8.1**][det-nix-version] is now available, including important security improvements. This release follows the revocation of [Determinate Nix 3.8.0][det-nix-3-8-0] after discovering a security regression for macOS users. All users on macOS should upgrade to Determinate Nix 3.8.1 as soon as possible to address the issue. We will publish further details in a follow-up post. Note that platform and security notices are also published to our [status page][status-page], and users are encouraged to subscribe for updates. For further questions, feel free to contact [support]. ## How to get Determinate Nix \{#install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.8.1 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.8.1" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - uses: DeterminateSystems/flake-checker-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [det-nix-3-8-0]: /blog/changelog-determinate-nix-380/ [det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.8.1 [det-nix]: https://docs.determinate.systems/determinate-nix [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.8.1/aarch64-linux [nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.8.1/x86_64-linux [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [status-page]: https://status.determinate.systems/ [support]: mailto:support@determinate.systems --- **Changelog: a faster `nix flake check`, improved flake locks, and lazy trees rolled out to 20% of users** Published: July 11, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-380 [Determinate Nix][det-nix] version [**3.8.0**][det-nix-version], based on version [2.30.0][nix-version] of upstream Nix, includes a variety of important improvements. ## An improved `nix flake check` \{#nix-flake-check} The [`nix flake check`][nix-flake-check] command no longer downloads flake outputs in cases when no building is necessary. This command validates that all of a flake's output successfully evaluate and that all outputs can thus be built. If the outputs are available in a binary cache, then both properties have already been confirmed to be true, and `nix flake check` shouldn't need to re-evaluate. Notably, downloading the output from the binary cache is not strictly necessary for the validation. That's because `nix flake check` never created any garbage collection roots, and therefore makes no guarantees about the build product being in the local store. Previously, `nix flake check` would download a flake output if the full build is available in a binary cache. Some users may find that this change significantly reduces costly bandwidth and CI workflow time. In our internal testing, we saw a reduction of several minutes in some of our repositories. ## Improved flake locking for transitive dependencies \{#transitive-dependencies} Determinate Nix now re-locks all transitive dependencies when changing a [flake input][inputs]'s source URL. This fixes an issue where Nix wouldn't re-lock those inputs in some scenarios and thus incorrectly used the old inputs' dependencies. ## A smarter `determinate-nixd upgrade` \{#determinate-nixd-upgrade} If you try to upgrade Determinate Nix when you're already on the latest version, Determinate Nixd now exits instead of upgrading again. This is a small improvement but a nice quality-of-life boost. Some users also reported `determinate-nixd version` timing out the first time it's run after a reboot. That should be fixed now as well. ## Lazy trees is rolled out to 20% of users \{#lazy-trees-rollout} In the last release, we rolled out [lazy trees][lazy-trees] to 5% of Determinate Nix users. After a successful 5% rollout, we're now expanding this group to 20% of users. You can see if you're enrolled in the lazy trees rollout using the [`determinate-nixd version`][dnixd-version] command: ```shell title="Check if Determinate Nix is due for an upgrade" determinate-nixd version ``` If Determinate Nix is up to date, you should see this: ```shell Determinate Nixd daemon version: 3.8.0 Determinate Nixd client version: 3.8.0 You are running the latest version of Determinate Nix. The feature lazy-trees is enabled. ``` If you see any bugs or would like to opt out, please contact [support@determinate.systems][support]. Lazy trees is a massive usability and performance improvement to flakes and we're excited to see it roll out to thousands more users daily. ## How to get Determinate Nix \{#install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.8.0 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.8.0" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - uses: DeterminateSystems/flake-checker-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [det-nix]: https://docs.determinate.systems/determinate-nix [det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.8.0 [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [dnixd-version]: https://docs.determinate.systems/determinate-nix#determinate-nixd-version [inputs]: https://zero-to-nix.com/concepts/flakes#inputs [lazy-trees]: https://dtr.mn/lazy-trees [nix-flake-check]: https://manual.determinate.systems/command-ref/new-cli/nix3-flake-check.html [nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.8.0/aarch64-linux [nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.8.0/x86_64-linux [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [support]: mailto:support@determinate.systems [nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.30 --- **Changelog: faster CI, deep flake overrides, and a better `nix store delete` in Determinate Nix 3.7.0** Published: July 4, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-370 Celebrate the Fourth of July with the release of [Determinate Nix][det-nix] version [**3.7.0**][det-nix-version]! We've shipped several Nix performance and UX improvements and we're also excited to roll out [lazy trees][lazy-trees] to more users. ## Prefetch flake inputs in parallel This release introduces the `nix flake prefetch-inputs` command to Determinate Nix. [Flake inputs][inputs] are typically fetched "just in time." That means that Nix fetches flake inputs as needed during the evaluation process, which can slow down evaluation in projects with many flake inputs. Ideally, though, Nix could avoid such slow-downs by fetching flake inputs **prior to evaluation**. The new `nix flake prefetch-inputs` command does precisely that, fetching all flake inputs in parallel so that they're already available when you need to evaluate any expression. We expect that running this new command before building things with Nix should dramatically improve evaluation performance for most projects, especially in CI. Note that projects with many unused flake inputs may not benefit from this change since the new command fetches every inputs whether they're used or not. We plan to run some experiments in GitHub Actions to see if and how much this helps. ## Deep flake input overrides now work as expected An override like this... ```nix { inputs.foo.inputs.bar.inputs.nixpkgs.follows = "nixpkgs"; } ``` ...used to implicitly set `inputs.foo.inputs.bar` to `flake:bar`, which led to an unexpected error like this: ```shell error: cannot find flake 'flake:bar' in the flake registries ``` With this release, Nix no longer creates a parent override (like for `foo.bar` in the example above) if it doesn't set an explicit `ref` or `follows` attribute; instead, Nix only recursively applies its child overrides. This closes four upstream Nix issues. Check out [DeterminateSystems/nix-src#95][nix-src-95] for more details and a snazzy diagram! ## `nix store delete` now shows you why deletion was not possible Here's some example error output: ```shell error: Cannot delete path '/nix/store/6fcrjgfjip2ww3sx51rrmmghfsf60jvi-patchelf-0.14.3' because it's referenced by the GC root '/home/eelco/Dev/nix-master/build/result'. error: Cannot delete path '/nix/store/rn0qyn3kmky26xgpr2n10vr787g57lff-cowsay-3.8.4' because it's referenced by the GC root '/proc/3600568/environ'. error: Cannot delete path '/nix/store/klyng5rpdkwi5kbxkncy4gjwb490dlhb-foo.drv' because it's in use by '{nix-process:3605324}'. ``` ## Lazy-tree improvements Improved lazy tree evaluation caching for flakes accessed with a `path` flakeref. ## Lazy trees on its way to general availability Since the last release, we've finished preparation for rolling out lazy tree to more users. We're going to start enabling lazy trees for about 5% of Determinate Nix users on version 3.7.0 outside of CI. You can see if you're enrolled in the lazy trees rollout using the [`determinate-nixd version`][dnixd-version] command: ```shell title="Check if Determinate Nix is due for an upgrade" determinate-nixd version ``` If Determinate Nix is up to date, you should see this: ```shell Determinate Nixd daemon version: 3.7.0 Determinate Nixd client version: 3.7.0 You are running the latest version of Determinate Nix. The feature lazy-trees is enabled. ``` If you'd like to opt out, please contact [support@determinate.systems][support]. We're excited for more folks to experience the, frankly, huge performance benefits of lazy trees in their workflows. ## How to get Determinate Nix \{#install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.7.0 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.7.0" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - uses: DeterminateSystems/flake-checker-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.7.0 [det-nix]: https://docs.determinate.systems/determinate-nix [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [dnixd-version]: https://docs.determinate.systems/determinate-nix#determinate-nixd-version [inputs]: https://zero-to-nix.com/concepts/flakes#inputs [lazy-trees]: https://dtr.mn/lazy-trees [nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.7.0/aarch64-linux [nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.7.0/x86_64-linux [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [support]: mailto:support@determinate.systems [nix-src-95]: https://github.com/DeterminateSystems/nix-src/issues/95 --- **Changelog: lazy trees and security improvements** Published: June 24, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-367 Determinate Nix [**v3.6.7**][det-nix-version] includes important security patches and users should [upgrade](#install) as soon as possible. ## Security contents This release fixes GHSA-g948-229j-48j3, a vulnerability reported by [Snyk]'s Security Labs team. The vulnerability allowed a user who could build software with the Nix daemon and coordinate running a program on the host to elevate to root. The upstream Nix project has not fully published the security advisory but we will update this post when they do. ## Lazy trees now saves NAR hashes to the `flake.lock` [Lazy trees][lazy-trees] now produces `flake.lock` files with NAR hashes. Here's an example command that exhibits this behavior: ```shell title="Generate a lockfile with NAR hashes" nix flake update --commit-lock-file ``` The previous behavior omitted NAR hashes from the `flake.lock`, which meant that adopting lazy trees required full buy-in from all users. Now, you can more confidently enable lazy trees in CI and other workflows without first enabling lazy trees for all users. You can restore the previous behavior by setting `lazy-locks` to `true`. ## Improved caching with impure evaluation and lazy trees Users have reported less effective caching when using lazy trees and impure evaluation. This release improves the caching side of things and should improve things substantially. ## Lazy trees on its way to general availability We have laid the ground-work to move lazy trees from feature preview into general availability. For the next phase in this process we will start progressively enabling lazy trees for users. If you'd like to opt out, please contact [support@determinate.systems][support]. You can see if you're enrolled in the lazy trees using the [`determinate-nixd version`][dnixd-version] command: ```shell title="Check if Determinate Nix is due for an upgrade" determinate-nixd version ``` If Determinate Nix is up to date, you should see this: ```shell Determinate Nixd daemon version: 3.6.7 Determinate Nixd client version: 3.6.7 You are running the latest version of Determinate Nix. The feature lazy-trees is enabled. ``` ## How to get Determinate Nix \{#install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.6.7 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.6.7" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our [NixOS ISO][nixos-iso] with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - uses: DeterminateSystems/flake-checker-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.6.7 [det-nix]: https://docs.determinate.systems/determinate-nix [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [dnixd-version]: https://docs.determinate.systems/determinate-nix#determinate-nixd-version [lazy-trees]: https://dtr.mn/lazy-trees [nixos-iso]: https://github.com/DeterminateSystems/nixos-iso/releases/tag/v3.6.7 [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [snyk]: https://labs.snyk.io/security-labs [support]: mailto:support@determinate.systems --- **FlakeHub now supports Semaphore CI** Published: June 19, 2025 URL: https://determinate.systems/blog/semaphore-ci Providing synchronized [development environments][envs] across continuous integration/delivery (CI/CD) platforms and developer workstations has always been one of Nix's most compelling use cases. And so we're excited to announce that [FlakeHub] now supports [Semaphore CI][semaphore] as our third major CI platform after [GitHub Actions][actions] and [GitLab CI][gitlab]. Semaphore is a powerful open source CI/CD platform that you can [run on your own infrastructure][semaphore-self-hosting] or [in the cloud][semaphore-cloud]. It features a [lovely UI][semaphore-canvas] for visualizing workflows, a built-in [observability suite][semaphore-observability], and much more. We encourage you to check it out. With this new support, you can now [publish flakes][publish], including [private flakes][private-flakes], to FlakeHub and push store paths to [FlakeHub Cache][flakehub-cache] inside your Semaphore CI runs. Here is an Semaphore workflow configuration that does both: ```yaml title=".semaphore/semaphore.yml" collapse={8-80} version: v1.0 name: Push store paths to FlakeHub Cache and publish flake release on FlakeHub agent: machine: type: f1-standard-4 os_image: ubuntu2404 blocks: - name: dependencies: [] task: prologue: commands: # Get the latest version of the repository's source code from GitHub - checkout jobs: - name: Publish flake and cache package commands: # The flake's repository - export FLAKEHUB_PUSH_REPOSITORY="$(echo "${SEMAPHORE_ORGANIZATION_URL}" | cut -d "." -f1 | cut -d '/' -f3)/${SEMAPHORE_PROJECT_NAME}" # Environment variables for Magic Nix Cache, which automatically pushes Nix artifacts to FlakeHub Cache - export MAGIC_NIX_CACHE_CLOSURE_URL="https://install.determinate.systems/magic-nix-cache-closure/branch/main/X64-Linux?ci=semaphore" - export MNC_LISTEN="127.0.0.1:37515" # Install Determinate Nix and start the Nix daemon - curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux --determinate --no-confirm --init systemd - . /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh # Log in using the `determinate-nixd login` command (used by magic-nix-cache, substitutions) - echo "${SEMAPHORE_OIDC_TOKEN}" | determinate-nixd login token --token-file /dev/stdin # Acquire the `flakehub-push` executable - curl -L "${FLAKEHUB_PUSH_BINARY_URL}" | sudo tee /usr/bin/flakehub-push &>/dev/null - sudo chmod +x /usr/bin/flakehub-push # Acquire the `magic-nix-cache` executable - export MNC_CLSR="$(curl -L "${MAGIC_NIX_CACHE_CLOSURE_URL}" | xz -d | sudo "$(which nix-store)" --import | tail -n1 | head -n1)" - sudo ln -sf "${MNC_CLSR}/bin/magic-nix-cache" /usr/bin/magic-nix-cache - magic-nix-cache --help # Stage login credentials for `flakehub-push` - export FLAKEHUB_PUSH_OIDC_TOKEN="${SEMAPHORE_OIDC_TOKEN}" # Start Magic Nix Cache - export MNC_STARTUP_FILE="/tmp/mnc-startup" - nohup magic-nix-cache --listen "${MNC_LISTEN}" --startup-notification-file "${MNC_STARTUP_FILE}" &>/tmp/mnc.log & - | ( STARTED=0 for n in {1..6}; do if [ -e "${MNC_STARTUP_FILE}" ]; then echo "magic-nix-cache daemon has successfully started up after ${n} attempt(s)" STARTED=1 break else echo "waiting on magic-nix-cache daemon; on attempt ${n}" sleep 2 fi done if [[ "${STARTED}" != "1" ]]; then echo "The daemon did not start up within 60 seconds; exiting" exit 1 fi ) || true # Build a package output by the repository's Nix flake - nix build ".#packages.x86_64-linux.default" # Publish a flake release to FlakeHub if and only if it's a tag reference - | if [[ "${SEMAPHORE_GIT_REF_TYPE}" == "tag" ]]; then flakehub-push \ --tag "$(cat "${SEMAPHORE_GIT_REF}" | cut -d '/' -f2)" \ --visibility private \ --include-output-paths fi # Stop Magic Nix Cache - curl -XPOST "http://${MNC_LISTEN}/api/workflow-finish" ``` For more detailed information, check out the [Semaphore CI guide][semaphore-guide] in the [Determinate documentation][docs]. If you run into any trouble, get in touch with us [on Discord][discord] or drop us an email at [support@determinate.systems][support]. Enjoy! [actions]: https://github.com/features/actions [discord]: https://determinate.systems/discord [docs]: https://docs.determinate.systems [envs]: https://zero-to-nix.com/concepts/dev-envs [flakehub]: https://flakehub.com [flakehub-cache]: https://flakehub.com/cache [gitlab]: https://docs.gitlab.com/ci [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [publish]: https://docs.determinate.systems/flakehub/publishing [semaphore]: https://semaphore.io [semaphore-canvas]: https://semaphore.io/product/canvas [semaphore-cloud]: https://semaphore.io/cloud [semaphore-guide]: https://docs.determinate.systems/guides/semaphore [semaphore-observability]: https://semaphore.io/product/metrics-and-observability [semaphore-self-hosting]: https://semaphore.io/product/self-hosted-agents [support]: mailto:support@determinate.systems --- **Changelog: macOS Tahoe, lazy trees, and docs in Determinate Nix 3.6.6** Published: June 18, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-366 We're excited to announce the release of [Determinate Nix][det-nix] version [**3.6.6**][det-nix-version], based on version [2.29.0][nix-version] of upstream Nix. ## Update on macOS Tahoe We've been testing our releases on macOS Tahoe internally since the first day of its release. All of our data shows that Determinate Nix works great on the current Tahoe beta. We'll continue to monitor the situation. If you see something that we should take a look at, please leave a comment on [this GitHub issue](https://github.com/DeterminateSystems/determinate/issues/100). ## Update on lazy trees We released [lazy trees][lazy-trees] in Determinate Nix 3.5.2, about a month ago, and since then hundreds of users have activated them. Across the board, these users have reported much faster CI times and in some scenarios a speedup of 5x or more. This 3.6.6 release includes additional performance and reliability improvements for lazy trees. It's easy to turn them on by adding this single line to your `nix.custom.conf`: ```shell title="/etc/nix/nix.custom.conf" lazy-trees = true ``` On NixOS, you can enable the NixOS option: ```nix title="/etc/nixos/configuration.nix" { nix.settings.lazy-trees = true; } ``` For those keeping track, you may have noticed that we skipped 3.6.3 and 3.6.4. We canceled the release of 3.6.3 and 3.6.4 after we identified some regressions around [FlakeHub authentication][flakehub-auth]. We've since improved our test suite to avoid this in the future. ## Even faster evaluation with lazy trees through improved input caching Some users reported that flake inputs were not being cached as often as they expected. [Eelco Dolstra][eelco] landed several pull requests to make this better, which has cut CI times for some users from over two minutes to under 20 seconds. ## Extended lazy trees test suite We now run the full Nix test suite and the Nixpkgs library test suite with lazy trees enabled. This revealed two bugs that are now fixed. ## `determinate-nixd status` now correctly checks the `netrc-file` setting on NixOS Running a [`determinate-nixd status`][dnixd-status] check examines Nix's `netrc-file` configuration setting. We added this check to help users who accidentally changed the setting, which breaks FlakeHub access. On NixOS, we unfortunately used the wrong path for Nix and the check didn't work. By the way: if you need to customize the netrc, use [`authentication.additionalNetrcSources`](https://docs.determinate.systems/determinate-nix#additionalnetrcsources) instead of changing the netrc file. ## Missing sandbox paths now have a sensible error message Users with a misconfigured [`sandbox-paths`][sandbox-paths] configuration used to see a mysterious error about a missing file. Users now get an error message that helps them solve the problem: ```shell path '/etc/nix/some-missing-file' is configured as part of the `sandbox-paths` option, but is inaccessible ``` ## The Determinate package on macOS can upgrade more systems This used to fail on systems that used `nix profile` to manage the default profile. We can now upgrade these systems as well. ## Assorted improvements * When remote building with `--keep-failed`, we only show the "you can rerun" message if the derivation's platform is supported on the current machine: [DeterminateSystems/nix-src#87](https://github.com/DeterminateSystems/nix-src/pull/87) * Lazy tree paths in messages are rendered without the `/nix/store/...` prefix in substituted source trees: [DeterminateSystems/nix-src#91](https://github.com/DeterminateSystems/nix-src/pull/91) * We fixed built-in functions being registered twice: [DeterminateSystems/nix-src#97](https://github.com/DeterminateSystems/nix-src/pull/97) * We fixed the link to `builders-use-substitutes` documentation for the `builders` option: [DeterminateSystems/nix-src#102](https://github.com/DeterminateSystems/nix-src/pull/102) * We updated error messages to remove what we call the "fake future tense:" [DeterminateSystems/nix-src#92](https://github.com/DeterminateSystems/nix-src/pull/92) * We added missing help text to some `determinate-nixd auth login` options. ## How to get Determinate Nix If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.6.6 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.6.6" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our [NixOS ISO][nixos-iso] with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - uses: DeterminateSystems/flake-checker-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.6.6 [det-nix]: https://docs.determinate.systems/determinate-nix [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [dnixd-status]: https://docs.determinate.systems/determinate-nix#determinate-nixd-status [eelco]: /people/eelco-dolstra [flakehub-auth]: https://docs.determinate.systems/flakehub/concepts/authentication [lazy-trees]: /blog/changelog-determinate-nix-352 [nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.29 [nixos-iso]: https://github.com/DeterminateSystems/nixos-iso/releases/tag/v3.6.6 [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [sandbox-paths]: https://nix.dev/manual/nix/latest/command-ref/conf-file.html#conf-sandbox-paths --- **Bringing back the Magic Nix Cache Action** Published: June 13, 2025 URL: https://determinate.systems/blog/bringing-back-magic-nix-cache-action A while back, we [announced][announcement] the end of life for our [Magic Nix Cache Action][mnca], a [GitHub Action][actions] that uses Actions' [built-in cache][gha-cache] to share the results of Nix builds between workflow runs. Nix users responded quite positively to getting a viable Nix binary cache for free and with zero configuration and we were happy to provide it to them. And then we were sad when GitHub announced that the Actions cache was being [deprecated][github-blog-post] in favor of a new (and undocumented) API. Today, we're happy to announce that **Magic Nix Cache Action is back**. GitHub user [jchv] submitted a heroic [pull request][v2-pr] to make [Magic Nix Cache][mnc] communicate with the new version of Actions' cache API. That means that as of the [most recent release][v2-release] of Magic Nix Cache you can once again use the Magic Nix Cache Action in your workflows. Here's an example configuration: ```yaml title="Example Magic Nix Cache Action workflow" {10} jobs: nix-build: runs-on: ubuntu-latest permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/magic-nix-cache-action@main - name: Build and cache with Nix run: nix build ``` ## Limitations Please be aware, though, that Magic Nix Cache Action continues to have some deep limitations: 1. It provides caching only between runs of a specific workflow in a specific repository and thus doesn't serve as a standard Nix binary cache available across a wide variety of clients. That means that you can't, for example, cache some Nix package builds in Actions and make those available to you and your teammates. 2. GitHub Actions' cache API is *not* a purpose-built Nix binary cache. Unsurprisingly, its performance is decent but not great. For most use cases, you should opt for [FlakeHub Cache][cache] instead, which offers true all-purpose binary caching and dramatically better performance and use Magic Nix Cache Action only when you want better performance between CI runs. ## An important caveat On a similarly sober note, we do need to make something clear. As I said above, the approach provided by [jchv] is based on a reverse engineering of GitHub's new caching API. GitHub has *not* published the [Protobuf] sources for that API and that means both that there's a good bit of guesswork here and that the API could change at any time, which could in turn make the new approach nonviable. We at Determinate Systems do hope that this will continue to work for years and years but the rug could in principle be pulled at any time. C'est la vie. We'd like to express our most sincere gratitude to [jchv] for their help. Happy (free) caching, everyone! [actions]: https://github.com/features/actions [announcement]: /blog/magic-nix-cache-free-tier-eol [cache]: https://flakehub.com/cache [gha-cache]: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows [github-blog-post]: https://github.blog/changelog/2024-09-16-notice-of-upcoming-deprecations-and-changes-in-github-actions-services [jchv]: https://github.com/jchv [mnc]: https://github.com/DeterminateSystems/magic-nix-cache [mnca]: https://github.com/DeterminateSystems/magic-nix-cache-action [protobuf]: https://protobuf.dev [v2-pr]: https://github.com/DeterminateSystems/magic-nix-cache/pull/139 [v2-release]: https://github.com/DeterminateSystems/magic-nix-cache/releases/tag/v0.1.5 --- **Changelog: docs, diagnostics, and resilience improvements** Published: June 5, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-362 [Determinate Nix][det-nix] version [**3.6.2**][det-nix-version], based on version [2.29.0][nix-version] of upstream Nix, includes a variety of small but substantial ergonomic improvements across a variety of domains. ## Migrate to or upgrade Determinate Nix *without* building it on NixOS [NixOS users][nixos] can now migrate to and upgrade Determinate Nix by fetching it from a cache rather than needing to build it from source. When initially installing Determinate Nix 3.6.2 using the [`determinate` flake][determinate-flake], make sure to pass a few extra flags to `nixos-rebuild`: ```shell title="Flags to pass when rebuilding on NixOS" {2-3} sudo nixos-rebuild \ --option extra-substituters https://install.determinate.systems \ --option extra-trusted-public-keys cache.flakehub.com-3:hJuILl5sVK4iKm86JzgdXW12Y2Hwd5G07qKtHTOcDCM= \ --flake ... \ switch ``` This uses `install.determinate.systems`, our artifact server, as a binary cache for fetching Determinate Nix. After the upgrade, users who are logged in to [FlakeHub] have [FlakeHub Cache][cache] in their substituters list. If you are *not* logged in, you have `install.determinate.systems` in your substituters list. This has been an annoying problem for quite some time because FlakeHub Cache requires authentication and offers no public, authentication-free caching. We solved this problem by teaching `install.determinate.systems` how to be an authenticated proxy to FlakeHub Cache. This public cache shows FlakeHub Cache's [access control rules][cache-auth] in action. ## Autocompletion for Determinate Nixd You can enable auto-completion for [Determinate Nixd][dnixd] using the new `determinate-nixd completion` subcommand. Pass your preferred shell—one of `bash`, `elvish`, `fish`, `powershell`, or `zsh`—and evaluate the result. Here's an example for zsh: ```shell title="Load shell completion script for zsh" eval "$(determinate-nixd completion zsh)" ``` Now when you run `determinate-nixd` and type in, say, "a" and hit **Tab**, it helpfully completes to `determinate-nixd auth`. ## Track down and fix "inefficient double copy" causes When we recently released [lazy trees][lazy-trees] in feature preview, we added a warning for when a Nix expression forces Determinate Nix to copy the flake's source to the store unnecessarily. Unfortunately, that warning was pretty useless because it didn't tell you *where* that happened. Determinate Nix now identifies where in the source code the extra file copying happens. So a command like `nix build .#extraCopy` would now produce a log message like this: ```shell title="Inefficient double copy message" {4} warning: Copying '/Users/grahamc/src/github.com/DeterminateSystems/samples/' to the store again You can make Nix evaluate faster and copy fewer files by replacing `./.` with the `self` flake input, or `builtins.path { path = ./.; name = "source"; }` Location: /Users/grahamc/src/github.com/DeterminateSystems/samples/extra-copy.nix:12:11 error: Cannot build '/nix/store/2hh3855rfsabcj4gm2nl65a9la7xbcih-extra-copy.drv'. ``` ## Eradicated symbols from `nix profile` Historically, `nix profile` commands like `nix profile history` would print out symbols like `ε` and `∅` from set theory. While these symbols *are* precisely defined, we don't want our users to *need* to brush up on mathematical logic to understand what is happening. So `nix profile history` now prints a much more understandable summary. Here's an example output for `nix profile history` with the current Determinate Nix: ```shell title="nix profile history output" Version 1 (2024-12-06): home-manager-path: (no version) added Version 2 (2025-02-25) <- 1: bar: (no version) added foo: (no version) added home-manager-path: (no version) removed ``` Now let's take a look at an older version of Nix. This command... ```shell title="nix profile history for Nix version 2.18.0" nix run github:NixOS/nix/2.18.0 -- profile history ``` ...would produce output like this: ```shell title="nix profile history with set theory symbols" Version 1 (2024-12-06): home-manager-path: ∅ -> ε Version 2 (2025-02-25) <- 1: bar: ∅ -> ε foo: ∅ -> ε home-manager-path: ε -> ∅ ``` We think the former is much more readily comprehensible to users. ## Clarify what `--keep-failed` plus remote building does When you run `nix build --keep-failed`, the build directory won't be deleted from the system. This is helpful for digging into the issue and solving the build problem. Unfortunately, users are surprised to find that the directory doesn't exist if the build was run remotely. The build directory was kept but it was kept *on the remote*. We updated the build failure message to help the user out: ```shell title="Example log message" copying path '/nix/store/cryiszlhhss2h40r3f9m60l92i88m2xd-builder-failing.sh' to 'ssh://localhost'... note: keeping build directory '/tmp/nix-shell.8G497s/nix-build-failing.drv-16/build' warning: `--keep-failed` will keep the failed build directory on the remote builder. If the build's architecture matches your host, you can re-run the command with `--builders ''` to disable remote building for this invocation. ``` ## Better diagnostics when trying to build a derivation on the wrong system type Sometimes a Nix expression inadvertently depends on a [derivation] that needs to build on a different system. When this happens, Determinate Nix now prints an error that includes the chain of derivations pulling in the mismatching build: ```shell {2-5} % nix build .#badSystemNested error: Cannot build '/nix/store/5vsaxi730yl2icngkyvn8wiflik5wfmq-bad-system.drv'. Reason: required system or feature not available Required system: 'bogus' with features {} Current system: 'aarch64-darwin' with features {apple-virt, benchmark, big-parallel, nixos-test} error: Cannot build '/nix/store/2gh9xc4bcq11vjxc80lw5hs3y4vnfvqy-nested-bad-system-bottom.drv'. Reason: 1 dependency failed. Output paths: /nix/store/5kzq0mlvxhll9vayw22a5dy8j259vpp4-nested-bad-system-bottom error: Cannot build '/nix/store/4lxwf00yrxnsraq00ga4dz4v5fxpk8pw-nested-bad-system-top.drv'. Reason: 1 dependency failed. Output paths: /nix/store/glb9vi6zp3j0dlc3j1g1i46pkwbq7gx0-nested-bad-system-top ``` This error clearly indicates that the `bad-system` derivation was pulled in by `bad-system-top` via `bat-system-bottom`. This is a dramatic improvement over the previous behavior, which excluded any context that would help identify the cause of the issue: ``` $ nix run github:NixOS/nix/2.18.0 -- build .#badSystemNested error: a 'bogus' with features {} is required to build '/nix/store/5vsaxi730yl2icngkyvn8wiflik5wfmq-bad-system.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test} ``` ## Easily find where import-from-derivation is used Import from derivation ("IFD") means using Nix to build a Nix expression that you then import. A common example of this is [Haskell.nix](https://github.com/input-output-hk/haskell.nix). IFD is a convenient "escape hatch" when packaging some language ecosystems, but comes at a serious cost of performance when using Nix. Because of the severe performance impact, we strongly discourage IFD and even prohibit its use on packages published on FlakeHub. One of our customers noted they are trying to cut back on how often they use IFD, but could not find *where* they used it. We have introduced a new flag to Determinate Nix, `--trace-import-from-derivation` which makes it easier to find: ```shell $ nix build --trace-import-from-derivation .#packages.aarch64-darwin.example warning: built '/nix/store/8r2a9yk318q9nv8j3aqbd86r73j183fn-generated-nix-expression.drv^out' during evaluation due to an import from derivation ``` From here we can inspect the derivation to find what caused IFD to take place. ### Dramatically improved performance with `nix store copy-sigs` We have increased the concurrency of `nix store copy-sigs`. Copying signatures is not a CPU-heavy task but it was previously running one copy operation per core. It now uses the `http-connections` setting to limit the concurrency, which is typically much higher, and defaults to 25. We also updated the documentation to include a description and examples. ### Improved robustness while collecting garbage The garbage collector would occasionally fail in the middle of cleaning up when encountering files that can not be deleted. While Determinate Nix users are less likely to experience this problem due to managed garbage collection, it now identifies and logs the issue, and continues with garbage collection as normal. ### Configurable `max-jobs` You can now change the `max-jobs` setting in `/etc/nix/nix.custom.conf`. Previously, this value was forced to `auto`. Now it defaults to `auto`, but you can update it to your own preference. ### Documented the `nix`-command replacement for `nix-store --query --deriver` The documentation for `nix-store --query --deriver` now mentions this `nix`-command based alternative: ```shell nix path-info --json ... | jq -r '.[].deriver' ``` ## No more "unauthenticated" errors with FlakeHub Cache The flake no longer unconditionally includes FlakeHub Cache in your list of active substituters. That means you won't see 403 errors whenever you build something. We fixed this issue a long time ago for macOS and Linux users, but forgot to delete it from the module for NixOS users. Fixed now! ## `determinate.enable` for easy migrations You can now add the [Determinate module][determinate-flake] to your global NixOS configuration and disable it on specific hosts. This is useful while migrating to Determinate across a large fleet. Thank you to GitHub user [getchoo] for [their PR][pr-97]. ## `nix.settings` works again on NixOS Determinate Nix manages its own `nix.conf` configuration, which can cause issues when trying to use `nix.settings` options on NixOS. The Determinate module now configures NixOS to write out your extra settings to [`/etc/nix/nix.custom.conf`][custom-conf] instead. Now you can use `nix.settings` and its structured configuration on NixOS without needing to adjust the module internals yourself. ## Determinate Nix Installer self-test [Determinate Nix Installer][installer] performs a self-test at the end to measure how well it did. Thanks to Josh Heinrichs from Shopify, the zsh self-test no longer takes over `/dev/tty`. ## How to get Determinate Nix If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.6.2 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.6.2" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos] or our [NixOS ISO][nixos-iso] with Determinate Nix pre-installed. On GitHub Actions: ```yaml title=".github/workflows/nix-ci.yaml" on: pull_request: workflow_dispatch: push: branches: - main jobs: nix-ci: runs-on: ubuntu-latest # Include this block to log in to FlakeHub and access private flakes permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-cache-action@main - uses: DeterminateSystems/flake-checker-action@main - run: nix flake check ``` In Amazon Web Services: ```terraform title="aws.tf" data "aws_ami" "detsys_nixos" { most_recent = true owners = ["535002876703"] filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } filter { name = "architecture" values = ["x86_64"] } } ``` [cache]: https://docs.determinate.systems/flakehub/cache [cache-auth]: https://docs.determinate.systems/flakehub/concepts/authentication [custom-conf]: https://docs.determinate.systems/determinate-nix#determinate-nix-configuration [derivation]: https://zero-to-nix.com/concepts/derivations [det-nix]: https://docs.determinate.systems/determinate-nix [det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.6.2 [determinate-flake]: https://github.com/DeterminateSystems/determinate [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [flakehub]: https://flakehub.com [getchoo]: https://github.com/getchoo [installer]: https://github.com/DeterminateSystems/nix-installer [lazy-trees]: /blog/changelog-determinate-nix-352 [nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.29 [nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos [nixos-iso]: https://github.com/DeterminateSystems/nixos-iso/releases/tag/v3.6.2 [pr-97]: https://github.com/DeterminateSystems/nix-src/pull/97 --- **Changelog: improved support for self-hosted GitHub Actions runners** Published: May 29, 2025 URL: https://determinate.systems/blog/changelog-improved-self-hosted-determinate-nix-action The [`determinate-nix-action`][determinate-nix-action], which we recently [announced][announcement], now works on most self-hosted CI environments in GitHub Actions. These once-problematic environments should now work just fine: * [Kubernetes] * [Actions Runner Controller][arc] (ARC) * [act] Previously, our Action used [Docker] as a process supervision daemon, but this proved to be overcomplicated and buggy in certain cases. The problem is that any Docker-oriented approach only works if the Docker daemon's root filesystem is identical to the execution environment's root filesystem—an assumption that doesn't always hold in reality. Our new method is much simpler. We now spawn the [Determinate Nix daemon][dnixd] in the background and shut it down when the job shuts down. This new approach works in all previous cases but also expands support for `determinate-nix-action` to a multitude of new CI environments. If you'd like to give it a try, this example Actions workflow should do the trick: ```yaml title=".github/workflows/check.yaml" jobs: check: name: Nix checks runs-on: ubuntu-latest permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - name: Install Determinate Nix uses: DeterminateSystems/determinate-nix-action@v3 - run: nix flake check -L ``` [act]: https://github.com/nektos/act [announcement]: /blog/determinate-nix-action [arc]: https://github.com/actions/actions-runner-controller [determinate-nix-action]: https://github.com/DeterminateSystems/determinate-nix-action [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [docker]: https://docker.com [kubernetes]: https://kubernetes.io --- **Determinate Nix Action** Published: May 28, 2025 URL: https://determinate.systems/blog/determinate-nix-action Shortly after releasing [Determinate Nix Installer][nix-installer], we released [`nix-installer-action`][nix-installer-action], a GitHub Action for installing Nix using Determinate Nix Installer, our next-generation installer written in [Rust] with support for seamlessly uninstalling Nix and more. This Action has successfully delivered Nix to millions of runners and thus served its purpose quite ably. Since the [launch] of [Determinate Nix][det-nix], `nix-installer-action` has provided the option of installing Determinate Nix by applying `determinate: true` to your YAML configuration. But `nix-installer-action` has an important drawback: its version isn't synced with the version of Determinate Nix, making it difficult to target Determinate Nix versions in CI runs. And so today we're announcing that we've created a new Action, [`determinate-nix-action`][action] that installs Determinate Nix both by default and with a synced version. That means that you can use, say, Determinate Nix version 3.6.1 by specifying this: ```yaml title="Targeted version" - uses: DeterminateSystems/determinate-nix-action@v3.6.1 ``` Here's a full example configuration: ```yaml title=".github/workflows/nix.yaml" {11,12,15} on: pull_request: push: branches: [main] jobs: build-nix-pkg: name: Build Nix package runs-on: ubuntu-latest permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - name: Build package using Determinate Nix run: nix build ``` Note that the `permissions` block is necessary if you intend to use [private flakes][private-flakes] and [FlakeHub Cache][cache]. And as an alternative to using `v3`, you can pin to a specific release, such as [`@v3.6.1`][latest] (the most recent release as of this writing). In terms of configuration, `determinate-nix-action` provides most of the same [parameters][configuration] that you're used to from `nix-installer-action`—such as `extra-conf` for providing custom Nix configuration—leaving behind only those that don't make sense in this new Action. If you do opt to pin to a specific version of `determinate-nix-action`, we strongly recommend using tools like [Dependabot] and [Renovate] to automatically bump the version when necessary, lest you end up running out-of-date versions of Determinate Nix. If you'd like to target specific versions of Determinate Nix, we do recommend switching to this new Action. For more on why you should consider pinning your Actions, GitHub has a good [primer][security-guide]. And as with all Actions, you can pin `determinate-nix-action` to a specific Git hash, such as [`441b9e401ac050c38a07d8313748c5c2d17e8aff`][exact-hash]: ```yaml title=".github/workflows/nix.yaml" - uses: DeterminateSystems/determinate-nix-action@441b9e401ac050c38a07d8313748c5c2d17e8aff ``` If you're currently using [`nix-installer-action`][nix-installer-action] to install Determinate Nix don't worry: nothing is changing on that end and you can still set `determinate: true` to install Determinate Nix: ```yaml title=".github/workflows/nix.yaml" {3} - uses: DeterminateSystems/nix-installer-action with: determinate: true ``` [action]: https://github.com/DeterminateSystems/determinate-nix-action [cache]: https://docs.determinate.systems/flakehub/cache [configuration]: https://github.com/DeterminateSystems/determinate-nix-action?tab=readme-ov-file#%EF%B8%8F%EF%B8%8F-configuration [dependabot]: https://github.com/dependabot [det-nix]: https://docs.determinate.systems/determinate-nix [exact-hash]: https://github.com/DeterminateSystems/determinate-nix-action/tree/441b9e401ac050c38a07d8313748c5c2d17e8aff [latest]: https://github.com/DeterminateSystems/determinate-nix-action/releases/tag/v3.6.1 [launch]: /blog/announcing-determinate-nix [nix-installer]: https://github.com/DeterminateSystems/nix-installer [nix-installer-action]: https://github.com/DeterminateSystems/nix-installer-action [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [renovate]: https://github.com/renovatebot/renovate [rust]: https://rust-lang.org [security-guide]: https://docs.github.com/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions#using-third-party-actions --- **Changelog: introducing lazy trees** Published: May 15, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-352 [**Lazy trees**](#importance) have been one of the most hotly requested Nix features for quite some time. They make Nix much more efficient in larger repositories, particularly in massive monorepos. And so we're excited to announce that lazy trees have landed in [Determinate Nix][det-nix] version **3.5.2**, based on version [2.28.3][nix-version] of upstream Nix. We're confident that this initial release of lazy trees should produce few issues for users. But for the sake of caution, lazy trees is available solely on an **opt-in basis** for the time being. To start using it today, first [install or upgrade](#upgrade-install) Determinate Nix and enable lazy trees in your custom Nix configuration: ```shell title="/etc/nix/nix.custom.conf" lazy-trees = true ``` Our co-founder [Eelco Dolstra][eelco] currently has a [pull request open][lazy-trees-upstream-pr] to get lazy trees into [upstream Nix][upstream] and we hope to see that merged soon. ## Why lazy trees are important \{#importance} In a nutshell, lazy trees provide **faster and less resource-intensive evaluation** in many standard usage scenarios. For evaluations inside of the [Nixpkgs] repo, for example, we've frequently seen reductions in wall time of **3x or more** and reductions in disk usage of **20x or more**—and occasionally reductions far beyond even this. With lazy trees enabled, Nix scopes file copying to what the specific expression demands—and nothing more. More specifically, Nix uses a **virtual filesystem** to gather the necessary file state prior to copying anything to the [Nix store][store], "mounting" lazy source inputs to this virtual filesystem at `/nix/store/`. This provides a much more parsimonious mode of operation that should quietly make just about everything in Nix better, from standard package builds to development environments to NixOS deployment and beyond. If you'd like to see this in action, try evaluating something in Nixpkgs yourself. [Install or upgrade Determinate Nix](#upgrade-install) and run this sequence of commands: ```shell title="Evaluate a derivation in Nixpkgs with lazy trees disabled" # Clone Nixpkgs git clone https://github.com/NixOS/nixpkgs cd nixpkgs # Evaluate the cowsay derivation with lazy trees disabled time nix eval \ --no-eval-cache \ .#cowsay # Make a meaningless change to the source tree echo "" >> flake.nix # Evaluate the cowsay derivation again time nix eval \ --no-eval-cache \ .#cowsay ``` Adding an empty line to `flake.nix` (a meaningless change) invalidates the evaluation cache and forces Nix to copy the **entire source tree** to the [Nix store][store], otherwise the second evaluation would be faster. Now the same sequence of steps with lazy trees: ```shell title="Evaluate a derivation in Nixpkgs with lazy trees enabled" # Enable lazy trees echo "lazy-trees = true" | sudo tee -a /etc/nix/nix.custom.conf # Make a meaningless change to the source tree echo "" >> flake.nix # Evaluate the cowsay derivation time nix eval \ --no-eval-cache \ .#cowsay # Make a meaningless change to the source tree echo "" >> flake.nix # Evaluate the cowsay derivation again time nix eval \ --no-eval-cache \ .#cowsay ``` You'll notice two things: 1. The evaluation should be substantially faster with lazy trees (if not, please let us know [via email][email] or [on Discord][discord]). 1. With lazy trees, you won't see any indication that Nix is copying the source tree to the Nix store. That's because it's able to determine, via evaluation, that only the sources for the [`cowsay`][cowsay] package are necessary to achieve its ends. Saving time is good. Also good: using less disk space. I'll provide an example. First, let's evaluate something in Nixpkgs using a `./tmp` subdirectory inside the current directory as a fresh [Nix store][store]: ```shell title="Disk space usage with lazy trees disabled" # Disable lazy trees echo "lazy-trees = false" | sudo tee -a /etc/nix/nix.custom.conf # Evaluate the cowsay package nix eval \ --no-eval-cache \ --store ./tmp \ .#cowsay # See how much disk space is used by the local Nix store du -sh ./tmp ``` On my machine, this yields **304 megabytes**. Now let's try the same thing with lazy trees enabled: ```shell title="Disk space usage with lazy trees enabled" # Delete the local Nix store to start from scratch sudo rm -rf ./tmp # Enable lazy trees echo "lazy-trees = true" | sudo tee -a /etc/nix/nix.custom.conf # Evaluate the cowsay package nix eval \ --no-eval-cache \ --store ./tmp \ .#cowsay # See how much disk space is used by the local Nix store du -sh ./tmp # Delete the local Nix store to clean up sudo rm -rf ./tmp ``` This yields **13 megabytes**. That's over 23 times less disk space. This specific number isn't generalizable across all situations, of course, but it does illustrate just how much more efficient Nix can become in a standard scenario when using lazy trees. ## CI comparisons To illustrate some of these differences in a CI environment, I've set up a GitHub repo at [DeterminateSystems/lazy-trees-comparisons][comparisons-repo]. The [workflow configuration][comparisons-workflow] shows the sequences of steps involved (similar to the steps [above](#importance)). As an example, check out [this CI run][comparisons-result], which evaluates the `stdenv` attribute of Nixpkgs on GitHub Actions' `ubuntu-latest` runner. The evaluation time without lazy trees: ```shell title="Evaluation time without lazy trees" real 0m10.787s user 0m3.655s sys 0m6.759s ``` Wall time without lazy trees: almost **11 seconds**. Disk usage without lazy trees: **433 megabytes**. Evaluation time with lazy trees: ```shell title="Evaluation time with lazy trees" real 0m3.500s user 0m1.762s sys 0m1.701s ``` Wall time without lazy trees: **3.5 seconds**. Disk usage with lazy trees: **11 megabytes**. Again, a substantial difference. This is, of course, just one result of one run, but it's consistent with [more general runs][comparisons-group] and we encourage you to make your own comparisons. ## New warning When using Determinate Nix with lazy trees, you may see warnings like this: ```shell title="Inefficient copy warning" warning: Performing inefficient double copy of path '«github:nixos/nixpkgs/0c0bf9c057382d5f6f63d54fd61f1abd5e1c2f63»/' to the store. This can typically be avoided by rewriting an attribute like `src = ./.` to `src = builtins.path { path = ./.; name = "source"; }`. ``` You'll see this in cases where you specify the root of a repo as the source for a derivation. When you use `./.` as the source for a derivation, Nix copies your repo to the Nix store under the name `/nix/store/-source`, which is then copied *again* as `/nix/store/--source`. Because of that, this is an inefficient way to specify your source: ```nix title="flake.nix" { myPackage = myDerivationFunction { src = ./.; }; } ``` This is a much more efficient way: ```nix title="flake.nix" { myPackage = myDerivationFunction { src = builtins.path { path = ./.; name = "my-package-source"; }; }; } ``` The `name` field can be whatever you like. In a flake, you can also use `self` instead of `./.`, as in: ```nix title="flake.nix" { myPackage = myDerivationFunction { src = self; }; } ``` Ideally, this wouldn't be necessary and using `./.` wouldn't produce any inefficiency. But unfortunately, changing this behavior would constitute a backwards-incompatible change that we are highly averse to introducing. ## How to use lazy trees in GitHub Actions \{#actions} If you'd like to try out lazy trees in [GitHub Actions][actions], you can use our [Determinate Nix Action][determinate-nix-action] and supply `lazy-trees = true` in your configuration: ```yaml title=".github/workflows/build-with-lazy-trees.yaml" {5,8} steps: - uses: actions/checkout@v4 - name: Install Determinate Nix with lazy trees enabled uses: DeterminateSystems/determinate-nix-action@v3 with: extra-conf: lazy-trees = true - name: Set up FlakeHub Cache uses: DeterminateSystems/flakehub-cache-action@main - name: Build default package output run: nix build . ``` ## How to upgrade or install Determinate Nix \{#upgrade-install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.5.2 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.5.2" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos-module]. To verify that you're on the most recent version of Determinate Nix at any time: ```shell title="Check Determinate Nix version" determinate-nixd version ``` And again, once you *are* on the most recent version, you can enable lazy trees in your custom Nix configuration: ```shell title="/etc/nix/nix.custom.conf" lazy-trees = true ``` On NixOS, enable the NixOS option: ```nix nix.settings.lazy-trees = true; ``` ## More to come [Determinate Nix][det-nix] has introduced a number of improvements to Nix in recent months, including [JSON logging][json-logging] and [automatic fixes for hash mismatches][hash-mismatches], but we're confident that the lazy trees release presents the most consequential change to date because it establishes a new, qualitatively improved baseline for Nix performance. But we are far from finished with improving Nix's performance. In the coming months, we intend to ship more improvements to Nix's evaluation performance, including: 1. Improving evaluation caching 1. Bringing [parallel evaluation][parallel-eval] to more Nix operations 1. Providing multi-threaded unpacking of flakes, both flakes from tarballs (like those from [FlakeHub]) and those from source forges like GitHub. Stay tuned! [actions]: https://github.com/features/actions [comparisons-workflow]: https://github.com/DeterminateSystems/lazy-trees-comparisons/blob/main/.github/workflows/comparison.yaml [comparisons-repo]: https://github.com/DeterminateSystems/lazy-trees-comparisons [comparisons-result]: https://github.com/DeterminateSystems/lazy-trees-comparisons/actions/runs/15051253370/job/42306364011 [comparisons-group]: https://github.com/DeterminateSystems/lazy-trees-comparisons/actions/runs/15051253370 [cowsay]: https://github.com/piuccio/cowsay [det-nix]: https://docs.determinate.systems/determinate-nix [determinate-nix-action]: https://github.com/DeterminateSystems/determinate-nix-action [discord]: https://discord.com/invite/a4EcQQ8STr [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [eelco]: /people/eelco-dolstra [email]: mailto:support@determinate.systems [flakehub]: https://flakehub.com [hash-mismatches]: /blog/changelog-determinate-nix-331#hash-mismatches [json-logging]: /blog/changelog-determinate-nix-331#json-logging [lazy-trees-upstream-pr]: https://github.com/NixOS/nix/pull/13225 [nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.28 [nixos-module]: https://docs.determinate.systems/guides/advanced-installation#nixos [nixpkgs]: https://github.com/NixOS/nixpkgs [parallel-eval]: /blog/parallel-nix-eval [store]: https://zero-to-nix.com/concepts/nix-store [upstream]: https://github.com/NixOS/nix --- **Changelog: deprecating channels and indirect flake references** Published: May 6, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-342 We're excited to announce the release of [Determinate Nix][det-nix] **3.4.2**, based on version [2.28.3][nix-version] of upstream Nix. In our [last Determinate Nix release][last] we announced a bunch of fun new stuff, like [JSON logging][json-logging] and a new set of capabilities to handle [hash mismatches][hash-mismatches]. This time around, the vibe is a bit different, as we've opted to deprecate two capabilities in Nix: * [Channels](#channels) as an abstraction for handling Nix inputs. * [Indirect flake references](#indirect-flake-refs) in flakes. ## Deprecating channels \{#channels} For a good chunk of Nix's life, [channels] were *the* way to do things. If you needed to evaluate expressions from a Nix project, such as [Nixpkgs], you added a channel for it and periodically updated that channel to see newer Git commits. Channels worked well enough but they always carried the grievous drawback that two systems looking at the same Nix code could "see" two different Nix expressions because their respective channels had been updated to different commits—which we believe goes firmly against the grain of Nix's worldview. There's an additional problem that every user on a system gets their own channels (including the root user) and the root user's channels are inherited by all users (but not the other way around)—another source of persistent confusion. For these and other reasons, we announced our intention to deprecate [channels] in [this issue][intent-channels]. This command, for example, would build different versions of [ponysay] from different commits of Nixpkgs on different machines *unless* those machines somehow managed to coordinate their channels: ```shell title="nix-build command involving an unpinned Nixpkgs" nix-build --expr "(import {}).ponysay" ``` Although projects like [niv] helped the situation by providing a basic [pinning] mechanism, Nix deserved to have the [reproducibility] problem resolved at a more fundamental level—Nix deserved [flakes]. From the beginning of our life as a company, we at Determinate Systems [have believed][flakes-manifesto] that [flakes] are the future of Nix for a variety of reasons, but foremost because they [pin][pinning] your Nix dependencies at all times using a [`flake.lock`][lockfile] file, overcoming the core deficit of channels. Now that [flakes have been stabilized][det-nix-launch] in Determinate Nix, we're ready to begin the process of **deprecating channels**. In Determinate Nix 3.4.2, this means two things: 1. The [`nix-channel`][nix-channel] command now emits a deprecation warning any time you use it. 1. Both the `nix` CLI and the `nix-build` tool now emit warnings whenever you perform any actions related to channels. This `nix-build` command, for example, still works: ```shell title="nix-build command emitting deprecation warning" nix-build -I nixpkgs=channel:nixos-24.11 '' -A ponysay ``` The [`nix-build`][nix-build] command no longer performs any important functions now that the unified Nix CLI has been stabilized in Determinate Nix. We still install it for you in case you still have `nix-build` invocations in your scripts but we intend to phase it out. But because it uses the `nixos-24.11` channel, it emits this warning: ```shell title="nix-channel deprecation warning" warning: Channels are deprecated in favor of flakes in Determinate Nix. Instead of 'channel:nixos-24.11', use 'https://nixos.org/channels/nixos-24.11/nixexprs.tar.xz'. See https://zero-to-nix.com for a guide to Nix flakes. For details and to offer feedback on the deprecation process, see: https://github.com/DeterminateSystems/nix-src/issues/34. ``` This unified `nix` CLI command also emits a similar deprecation warning because it uses a channel: ```shell title="Unified CLI command emitting deprecation warning" nix build --file '' --nix-path nixpkgs=channel:nixos-24.11 ponysay ``` What we haven't yet deprecated is [lookup paths][lookup-paths] like ``. So this command works without emitting a warning: ```shell title="Unified CLI command emitting no warning" nix build --file '' ponysay ``` But it's important to note that it works because Determinate Nix's default settings contain an `extra-nix-path` setting that sets the value of `` to a flake reference: ```shell title="/etc/nix/nix.conf" extra-nix-path = nixpkgs=flake:https://flakehub.com/f/DeterminateSystems/nixpkgs-weekly/*.tar.gz ``` Commands like this also still work because they implicitly refer to the lookup value set in `extra-nix-path`: ```shell nix build --impure --expr '(import {}).cowsay' ``` While we don't necessarily encourage using the `` lookup path for *new* things, we know that many of you may have bits of Nix logic that use references like this and we want to continue to support this use case for now. ## Deprecating indirect flake references \{#indirect-flake-refs} Nix with flakes has, from its inception, offered a [flake registry][registry] that enables you to map *indirect flake references* to real flake references, such as `nixpkgs` to `https://flakehub.com/f/NixOS/nixpkgs/*`. With this mapping in place, these two commands would be equivalent: ```shell # With full reference nix flake show "https://flakehub.com/f/NixOS/nixpkgs/*" # With implicit reference nix flake show nixpkgs ``` The flake registry, which you can manipulate using the [`nix registry`][nix-registry] command, also enables you to use implicit references in flakes themselves. In this example, the URL for the `nix` flake input maps to whatever is in the flake registry: ```nix title="flake.nix" {2} { inputs.nix.url = "nix/2.28.3"; outputs = { self, nix, ... }: { # your outputs }; } ``` This capability also enables you to forgo specifying inputs entirely: ```nix title="flake.nix" {2} { outputs = { self, nixpkgs, ... }: { # your outputs }; } ``` We at Determinate Systems think that the former use case for implicit flake references—providing "shortcut" CLI commands—is quite reasonable. But we don't think that indirect references inside of flakes are reasonable. In fact, we've seen this as an **ill-conceived misfeature** for years because it produces a hole in flakes' reproducibility guarantees, as the reference can resolve to different things depending on what's in the registry. Although this *becomes* less of an issue when the indirect reference is resolved to a direct reference and written to the `flake.lock` file, indirect references still run against the spirit of flakes. We announced our intention to deprecate indirect flake references in `flake.nix` files in [this issue][intent-flakerefs], and in Determinate Nix 3.4.2 we've made some changes to that effect. This command still works... ```shell title="Indirect flake references on the command line" nix run nixpkgs#cowsay -- Moooo ``` ...but you will see a warning if you run commands against this `flake.nix` file... ```nix title="flake.nix" { outputs = { self, nixpkgs, ... }: { # your outputs }; } ``` ...as well as this one: ```nix title="flake.nix" { inputs.nixpkgs.url = "nixpkgs"; outputs = { self, nixpkgs, ... }: { # your outputs }; } ``` The warning looks like this, providing a hint to how you can address the issue: ```shell title="Indirect flake reference warning" warning: Flake input 'nixpkgs' uses the flake registry. Using the registry in flake inputs is deprecated in Determinate Nix. To make your flake future-proof, add the following to '/nix/store/96y77qcpivmvskagnd94hgqczrgcq2w6-source/flake.nix': inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable"; For more information, see: https://github.com/DeterminateSystems/nix-src/issues/37 ``` In the future we intend to completely prohibit indirect flake references in flakes. ## How to upgrade or install \{#upgrade-install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.4.2 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.4.2" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos-module]. ## Implications Sometimes less really is more. Here at Determinate Systems, we're on a mission to modernize Nix and make it not just suitable but rather indispensable for a wide range of enterprise use cases. Sometimes, that will mean adding scintillating new features that expand the range of what Nix can do. Other times, like today, it will mean engaging in some careful pruning. [channels]: https://zero-to-nix.com/concepts/channels [det-nix]: https://docs.determinate.systems/determinate-nix [det-nix-launch]: /blog/determinate-nix-30 [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [flakes]: https://zero-to-nix.com/concepts/flakes [flakes-manifesto]: /blog/experimental-does-not-mean-unstable [hash-mismatches]: /blog/changelog-determinate-nix-331#hash-mismatches [intent-channels]: https://github.com/DeterminateSystems/nix-src/issues/34 [intent-flakerefs]: https://github.com/DeterminateSystems/nix-src/issues/37 [json-logging]: /blog/changelog-determinate-nix-331#json-logging [last]: /blog/changelog-determinate-nix-331 [lockfile]: https://zero-to-nix.com/concepts/flakes#lockfile [lookup-paths]: https://nix.dev/manual/nix/latest/language/constructs/lookup-path [nix-build]: https://nix.dev/manual/nix/latest/command-ref/nix-build [nix-channel]: https://nix.dev/manual/nix/latest/command-ref/nix-channel [nix-registry]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-registry [nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.28 [nixos-module]: https://github.com/determinatesystems/determinate?tab=readme-ov-file#installing-using-our-nix-flake [nixpkgs]: https://github.com/NixOS/nixpkgs [niv]: https://github.com/nmattia/niv [pinning]: https://zero-to-nix.com/concepts/pinning [ponysay]: https://github.com/erkin/ponysay [registry]: https://zero-to-nix.com/concepts/flakes/#registries [reproducibility]: https://zero-to-nix.com/concepts/reproducibility --- **Changelog: JSON logging, a new experience around hash mismatches, and more** Published: April 16, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-331 Logging provides a window into what a program is up to, and that window is never more crucial than when something has gone wrong and you need granular insight into what has happened and how you can fix it. In Nix, logging is generally most important during the [realisation] process, and *ideally* Nix would provide world-class build error logs. But historically this has *not* been seen as one of Nix's strengths. We've set out to change that, and we're excited to announce that with the 3.3.1 release [Determinate Nix][det-nix] now supports [**JSON logging**](#json-logging) for a small subset of error logs (with a larger subset to come soon). Adding this capability to Nix has also enabled us to dramatically upgrade the user experience around things like fixing [hash mismatches](#hash-mismatches). Check out this video from Determinate Systems CEO and cofounder [Graham Christensen][graham] for a quick overview: Once you've done that, read on for some more in-depth information. ## JSON logging With Determinate Nix 3.3.1, Nix can now log some error messages as structured JSON. We've added this because JSON is generally more straightforward to parse by external tools. At this stage, Nix provides JSON logs in one specific case: when Nix encounters a [hash mismatch](#hash-mismatches) during a build. In the future we plan to enable JSON logging in more places but we think this is a good place to start. ## Automatic fixes for hash mismatches \{#hash-mismatches} Seasoned users of Nix are likely all-too familiar with this log output: ```shell title="Hash mismatch log output" error: hash mismatch in fixed-output derivation '/nix/store/2ybj73ri0dc9j28yzqlp7r1l2lpxnsxj-my-package.drv': specified: sha256-tmNn6/9jn/ansLI0QTfy6/fItRmBCOhb5AUOXbIWi24= got: sha256-GzabkpW8L2UH4rzBkFdv61RoJl8DQlICqZvY9Ql1MAU= ``` This certainly *does* provide the information needed to fix the problem. But in order to actually extract the correct hash—the information you really want here—you need to parse this log *as a string*. You can do this with [grep] and other tools but let's face it—it isn't a whole lot of fun. With JSON logging, however, Nix produces this instead: ```json title="Hash mismatch JSON output" { "v": "1", "c": "HashMismatchResponseEventV1", "drv": "/nix/store/2ybj73ri0dc9j28yzqlp7r1l2lpxnsxj-my-package.drv", "bad": [ "sha256-tmNn6/9jn/ansLI0QTfy6/fItRmBCOhb5AUOXbIWi24=" ], "good": "sha256-GzabkpW8L2UH4rzBkFdv61RoJl8DQlICqZvY9Ql1MAU=" } ``` You can parse JSON logs with tools like [jq] instead of, say, grep with a set of gnarly regular expressions. ### Fix hashes using Determinate Nixd \{#determinate-nixd-hashes} [Determinate Nixd][dnixd], which comes installed with [Determinate Nix][det-nix], enables you to automatically fix mismatched hashes: ```shell title="Apply hash fixes using Determinate Nixd" determinate-nixd fix hashes ``` Let's say that you've just run `nix build` and gotten an error because, say, a vendor hash in a derivation building a [Go] application is wrong. If you run the command above, Determinate Nixd will prompt you to ask if you'd like to fix hash mismatches for you. If you say **yes**, Determinate Nixd fixes the relevant files in place; all you need to do is commit the changes and you're ready to go. ### Identify hash mismatches in GitHub Actions While fixing hashes in your local development flows is nice, pretty much all of us forget to do this until it's too late and our continuous integration workflows are unhappy with us. But with JSON logging now in Determinate Nix, we offer a solution to hash mismatches at that level as well. If you'd like substantially improved hash mismatch *reporting*, all you have to do is use our [determinate-nix-action] for GitHub Actions. ```yaml title="Install Determinate Nix in GitHub Actions" {3} - uses: DeterminateSystems/determinate-nix-action@v3 ``` When you do that, you now get a GitHub-native notification rather than needing to dig through Nix error logs. This screenshot shows an example: Note that this is now surfaced directly in the Git diff UI—and in an attention-grabbing red color to boot. ### Automatic hash mismatches fixes in GitHub Actions To finish drawing the owl, run `determinate-nixd fix hashes --auto-apply` on failure to fix pull requests automatically: ```yaml title="Automatically correct hash mismatches" {3-4,12,16-27} jobs: build: permissions: contents: write steps: - uses: actions/checkout@v4 with: ref: ${{ github.head_ref }} - uses: DeterminateSystems/determinate-nix-action@v3 - run: nix flake check -L - name: Fix hash mismatches if: failure() run: | determinate-nixd fix hashes --auto-apply if ! git diff --quiet; then git config user.name "github-actions[bot]" git config user.email "41898282+github-actions[bot]@users.noreply.github.com" git add . git commit -m "[dependabot skip] Fix hashes" git push fi ``` ## Ergonomic fixes In addition to [JSON logging](#json-logging) and the new features that it has opened up, we've made some smaller improvements that we hope you'll like as well. ### Better suggestions for untracked files \{#untracked-files} When you use [flakes], Nix needs to work closely with Git to function properly. Sometimes there are untracked changes in your repository that keep Nix from working as expected. For years, Nix has emitted this log message when that happens: ```shell title="Previous error message for untracked changes" error: path '/nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source/flake.nix' does not exist ``` That's... pretty opaque. It's not clear why that file existing is somehow important and it doesn't tell you how to fix the problem. With Determinate Nix 3.3.1, you now see an error message like this instead: ```shell title="New error message for untracked changes" error: Path 'flake.nix' in the repository "/home/justme/my-project" is not tracked by Git. To make it visible to Nix, run: git -C "/home/justme/my-project" add "flake.nix" ``` Nix now provides not just a better error message but also actionable information that you can use to remedy the situation. This has been a paper cut for Nix users for *years* and we're really happy to put it behind us for good. ### `nix profile install` is now `nix profile add` \{#nix-profile-add} The [`nix profile`][nix-profile] command has an [`install`][nix-profile-install] subcommand that you can use to add a package to the current Nix profile. This command works as expected but we've always felt like "install" isn't really the right verb—and many of our users have shared that sentiment. In Determinate Nix 3.3.1, this has been replaced with an `add` subcommand: ```shell title="The new command to add a package to the current profile" nix profile add nixpkgs#ponysay ``` Running `nix profile install` still works as an alias, but we do recommend switching any scripts over to the `add` subcommand now. Like the [untracked files error message](#untracked-files), this isn't an earth-shattering change but it still feels good to bring Nix more in line with users' expectations. ## How to upgrade or install \{#upgrade-install} If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.3.1 with one [Determinate Nixd][dnixd] command: ```shell title="Upgrade command for version 3.3.1" sudo determinate-nixd upgrade ``` If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer: On Linux: ```shell title="Install Determinate Nix on Linux" curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` On NixOS, we recommend using our [dedicated NixOS module][nixos-module]. ## More to come Nix now has the necessary code paths to output JSON logs wherever they may be needed. We've started modestly with adding such logs, but even this modest beginning provides some powerful new workflows for Nix. We have some ideas for other areas where JSON logs can open up new paths in Nix and we'll be exploring those soon. On top of that, we've provided some [ergonomic changes](#ergonomic-fixes) that address small but long-standing and substantial user experience issues in Nix. And we plan to tackle a few more issues along these lines in the coming weeks. That's it for now! Stay tuned here on the blog for more advancements in the whole experience around using Nix. [det-nix]: https://docs.determinate.systems/determinate-nix [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [flakes]: https://zero-to-nix.com/concepts/flakes [go]: https://golang.org [graham]: /people/graham-christensen [grep]: https://www.gnu.org/software/grep/manual/grep.html [jq]: https://jqlang.org [determinate-nix-action]: https://github.com/DeterminateSystems/determinate-nix-action [nix-profile]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-profile [nix-profile-install]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-profile-install [nixos-module]: https://github.com/determinatesystems/determinate?tab=readme-ov-file#installing-using-our-nix-flake [realisation]: https://zero-to-nix.com/concepts/realisation --- **Best practices for Nix at work** Published: March 24, 2025 URL: https://determinate.systems/blog/best-practices-for-nix-at-work Last week, we at Determinate Systems held our first webinar, [Best practices for Nix at work][past], presented by [yours truly][me]. The session was a lot of fun, there were some great questions, and we're really excited to continue to do more. It was so fun, in fact, that we'll be doing another one next week, on **Thursday, March 27th at 1 pm GMT-3**, titled [Up and running with FlakeHub Cache][upcoming]. It will cover some conceptual basics around binary caching in Nix, what sets [FlakeHub Cache][cache] apart from other offerings, and how to best take advantage of FlakeHub Cache's unique feature set. I hope to see you there! In this post, I'd like to share some of the best practices that I covered in our [initial webinar][past]. As in the talk, I've split those into three sections: - [Best practices for using flakes](#flakes) - [Security best practices for Nix](#security) - [Best practices for organizational adoption](#adoption) ## Best practices for using flakes \{#flakes} At Determinate Systems, we've made it [abundantly clear][stable] that we believe that [flakes] are the future of Nix. So I won't cover *why* you should use flakes here, but I will lay out some best practices for *how* you should use flakes based on our deep experience as a company the last few years. ### Use SemVer \{#semver} On their own, flakes don't have a concept of [semantic versioning][semver] (SemVer). That's because most current Nix users pull flakes, such as [Nixpkgs], from source forges like [GitHub]. And since those source forges don't themselves have any concept of semantic versioning, at least when it comes to serving tarballs, Nix doesn't have the requisite information to work with. Other sources for flakes, like the local filesystem, also don't have the appropriate "hooks" that Nix would need to support SemVer. We at Determinate Systems have added SemVer to flakes with [FlakeHub], and unsurprisingly we do recommend using it. SemVer in Nix can apply not just to CLI tools and standard packages, but also to things like running services. Why not publish version 1.4.0 of your authentication service or 0.2.5 of your main web application? You can even use Nix adoption as a reason to finally SemVer-ify your stack. With SemVer, your Nix updates can become much smoother. Rather than updating a monolithic dependency like [Nixpkgs] every however-many months, you can make your updates much more granular, handling any issues with discontinuity in a much more gradual way. And if a flake outputs many different versioned things? See more about that [below](#many-flakes). ### Split things up into as many flakes as you need \{#many-flakes} In the past, the Nix community has generally gravitated toward a usage model that involves having few dependencies. We see this centralizing tendency in many corners and frequently encounter organizations that depend only on [Nixpkgs] in all of their projects. But things don't need to be so centralized. [Flakes] are a great mechanism for dividing Nix things up into smaller units, and we built [FlakeHub] to make flakes "cheap"—cattle, not pets, if you will. We built the [`flakehub-push`][flakehub-push] Action to make this trivial. And don't forget that you can put several flakes in a single repository, making this model friendly to large repos or even monorepos. So *how many* flakes do you need? There's no hard-and-fast rule here, but in general we recommend publishing **one flake per versioned thing**, where a "thing" could be, for example, a versioned package like a CLI tool or [NixOS] configurations for a service or a series of [Home Manager][hm] configurations. I might even venture to say that when in doubt, just create another flake. You can always consolidate into larger flakes later if need be. ### Update your inputs often \{#update-inputs} Keeping your inputs fresh can help make your updates smoother. It can make the difference between fixing a series of small things incrementally over time and spending an afternoon trying to disentangle a mess where you're not sure what is breaking what. We have two tools that can help with this: * Our [Flake Checker Action][flake-checker] helps you keep your [Nixpkgs] inputs up to date and either warns you or straight-up fails your CI runs if Nixpkgs is more than 30 days old (or a different number of days if you wish). * Our [update-flake-lock] Action automatically updates your flake inputs on your preferred cron schedule and creates a pull request. ### Avoid flake helper libraries if possible While we do encourage splitting things up into flakes, we generally recommend not using flake helper libraries like [`flake-utils`][flake-utils] and [`flake-parts`][flake-parts]. There is nothing intrinsically *wrong* with these libraries—they're made by Nix veterans and they're well built and maintained—but any time you introduce a flake input into your dependency tree, it's then an input to anyone depending on your flake (and anyone depending on that, and so on). And so you should strive to eliminate unnecessary inputs when possible, and we think that these are generally not necessary. Let's start with `flake-utils`, which provides some helpers for system-specific flake outputs. A worthy goal, certainly, but a few lines of Nix code will get you the same thing. At Determinate Systems, for example, we have a few lines of boilerplate that we include in every flake to handle system-specific outputs. The lines look something like this: ```nix title="flake.nix" let supportedSystems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ]; forEachSupportedSystem = f: nixpkgs.lib.genAttrs supportedSystems (system: f { pkgs = import nixpkgs { inherit system; }; }); in { ... } ``` Then we can use that for things like package outputs: ```nix title="flake.nix" { packages = forEachSupportedSystem ({ pkgs }: { client = ...; server = ...; }); } ``` `flake-parts` has more ambitious goals, striving to make flakes more module driven, but at the cost of, to my eyes at least, making flakes [more verbose][flake-parts-example] and convoluted. At Determinate Systems, we always prefer plain old Nix code or Nixpkgs `lib` functions and have always managed without abstractions like this. If you truly need `flake-utils` or `flake-parts`, please don't hesitate to use them. Just be aware of the cost that flake helpers like this incur and judge accordingly. If you find yourself relying on them in ways that can be readily replaced by a few lines of copy/pasted code, we say go with that. ## Security best practices for Nix \{#security} One of the great promises of Nix is that it's supposed to make building and deploying things more secure by eliminating some common sources of non-reproducibility. But Nix isn't magic here and there are still some basic precautions you should follow. ### Keep secrets out of the Nix store \{#secrets} Always keep in mind that the [Nix store][store] isn't a typical filesystem for running ad-hoc operations like reading, writing, copying, and deleting files. Instead, it's basically an instrument that Nix uses to function properly, and when you write something to the Nix store, it can be quite non-trivial to get it out. And so you should engage in whatever practices necessary to prevent yourself from writing secrets to any Nix store—especially if that Nix store happens to be a cache like [FlakeHub Cache][cache]. One common practice in the Nix community is to use tools like [agenix] and [sops-nix] for managing secrets. These are really solid tools and we think that they're fine for smaller organizations or deploying a small set of NixOS machines for personal use. But for larger organizations, we recommend going beyond this and avoiding static secrets entirely—or at least whenever possible. That could mean using [Vault] or an analogous service. ### Follow the principle of least access \{#least-access} In larger organizations, we recommend a general policy of keeping your Nix things private and then granting access to them only when necessary. For flakes, that means keeping your flakes [private][private-flakes] unless you absolutely need to distribute something to the Nix-using public. This ensures that only people inside your org can run operations like `nix build` or even `nix flake show` against those flakes. For caching, this means avoid public caches like [cache.nixos.org][cno] whenever possible and using trusted private caches like [FlakeHub Cache][cache] instead. Public caches have lots and lots of good stuff in them, of course, but they're massive and the security and review practices behind them are not always air tight. When you pull from them to build production systems, you're working with a porous trust model. We built FlakeHub Cache as a strictly private cache with no public access whatsoever to fill this precise gap in traditional approaches to Nix caching. ## Best practices for organizational adoption \{#adoption} Regardless of how wise the best practices mentioned above may be, they don't mean much if you can't get your organization up and running with Nix. While there is no one correct path to Nix adoption, we do have a few rules of thumb worth drawn from our experience working with organizations of many sizes. ### Spread knowledge While we and many others have been working hard to make Nix more straightforward to understand and use, it can be a challenge to get up to speed even with some conceptual basics like knowing what Nix *even is* and what benefits it can provide. The more knowledge about Nix that you can spread within your organization, the better. We created [Zero to Nix][z2n] as a flake-first documentation source for beginners, and we do strongly recommend passing that along to your coworkers. There are also some great official resources, such as [nix.dev]. ### Adopt a gradual approach \{#gradual} Bringing Nix into any organization can be disruptive in a variety of ways. But because Nix can transform so many things, there are lots of ways to gently expand usage. Let's say that you're using Nix on just one team and only for [development environments][envs]. Just one team can be enough to demonstrates gains—faster CI runs, more synchronized environments, fewer "works on my machine" mishaps—significant enough to compel other teams to take action. So you can talk a second team into using Nix for development environments. And talk the first team into using Nix for packaging their tools and distributing them to other teams. And maybe talk a third team into converting their `Dockerfile`-driven services into [NixOS] services. Or you can convert all of the Python stuff in your org over to using Nix and then talk the PHP folks into doing the same. And so on. Ideally, from our perspective, you would wake up one morning with your entire org safely and happily under the Nix umbrella, but we know that things don't work that way. Fortunately, there are many paths to expanding internal adoption in the meantime. ## See you soon? This set of best practices isn't super systematic but it does provide a taste of what we went over in the webinar and what kind of content you can expect in the future there. We hope to see you [at the next one][upcoming] on FlakeHub Cache this week! [agenix]: https://github.com/ryantm/agenix [cache]: https://flakehub.com/cache [cno]: https://cache.nixos.org [envs]: https://zero-to-nix.com/concepts/dev-env [flake-checker]: https://github.com/DeterminateSystems/flake-checker-action [flake-parts]: https://flake.parts [flake-parts-example]: https://flake.parts/getting-started.html#existing-flake [flake-utils]: https://github.com/numtide/flake-utils [flakehub]: https://flakehub.com [flakehub-push]: https://github.com/DeterminateSystems/flake-checker [flakes]: https://zero-to-nix.com/concepts/flakes [github]: https://github.com [hm]: https://flakehub.com/flake/nix-community/home-manager [me]: /people/luc-perkins [nixos]: https://github.com/NixOS/nixpkgs/tree/master/nixos [nixpkgs]: https://github.com/NixOS/nixpkgs [nix.dev]: https://nix.dev [past]: https://lu.ma/87xuszxp [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [semver]: https://docs.determinate.systems/flakehub/concepts/semver [sops-nix]: https://github.com/Mic92/sops-nix [stable]: /blog/experimental-does-not-mean-unstable [store]: https://zero-to-nix.com/concepts/nix-store [upcoming]: https://lu.ma/ouooinmw [update-flake-lock]: https://github.com/DeterminateSystems/update-flake-lock [vault]: https://vaultproject.io [z2n]: https://zero-to-nix.com --- **Determinate Nix 3.0** Published: March 5, 2025 URL: https://determinate.systems/blog/determinate-nix-30 Today, we're thrilled to announce the release of [Determinate Nix 3.0][det-nix]. Determinate Nix is our professionally maintained, security-focused distribution of [Nix] designed for organizations that cannot compromise on security, compliance, or performance. The version number matters. The Nix project has long intended to release version 3.0 when [flakes] are stable. With Determinate Nix 3.0, we've fulfilled that promise by offering a **formal stability guarantee for flakes**, making them production ready today. ## Why flake stability matters Flake stability is essential for organizations developing mission-critical systems. Although flakes enhance the core features of Nix—reproducible environments, predictable builds, composable configurations—by enforcing [pinned][pinning] dependencies, we continue to see deep uncertainty in the ecosystem due to flakes' experimental status in upstream Nix. Without such a guarantee, organizations have expressed concerns about unpredictable breaking changes that could disrupt CI/CD pipelines, developer workflows, and production deployments. Our flake stability guarantee mitigates this risk, ensuring that your flake-based configurations function reliably through every update and enabling you to confidently adopt flakes for your most important projects today. ## Not a fork, a future-proof foundation Determinate Nix 3.0 isn't a fork of Nix—it's a downstream distribution that ensures full compatibility with the broader Nix ecosystem while incorporating the governance, security features, and performance optimizations necessary for mission-critical deployments. ## Building a better Nix Determinate Nix 3.0 represents a fundamental shift in how we deliver Nix to organizations with mission-critical requirements. We don't just package upstream Nix—we build it ourselves from source in our secure, [SLSA build level 3][slsa-3], controlled infrastructure. This independent build process is crucial for the trust, security, and stability that regulated environments demand. By maintaining our distribution, we can innovate at the pace our customers need without being constrained by upstream development cycles. This means that Determinate Nix customers will soon have immediate access to critical features like: - [**Parallel evaluation**][parallel-eval-pr] of Nix expressions, which can deliver 3-4x faster performance for complex configurations. - [**Lazy trees**][lazytrees-pr], which can dramatically improve evaluation speed for large source repositories. Our approach combines the best of both worlds: Determinate Nix customers benefit from cutting-edge capabilities today while we maintain compatibility with the broader Nix ecosystem and actively contribute our improvements to upstream Nix, where the community ultimately determines their inclusion. Choosing Determinate Nix gives you priority access to innovations that address real-world needs while ensuring seamless integration with existing systems. ## Security that meets modern standards Security is central to every aspect of Determinate Nix and the entire [Determinate platform][determinate]. - [**SOC 2 Type II certified**][soc2]: Our infrastructure meets rigorous compliance standards. - **Zero-trust security model**: Federated authentication combined with fine-grained access controls with federated authentication. - **Modern authentication**: Shift from static secrets to secure, policy-driven identities through Identity and Access Management (IAM) roles and identity providers such as [GitHub][github-auth] and [Microsoft Entra][entra-auth]. - **Corporate network integration**: Automatic certificate handling for [ZScaler], [Fortinet], and other security platforms. - **Defined security response**: We provide an SLA for vulnerability management, ensuring prompt and predictable security updates. - **Controlled build environments**: We build every component in our security-certified infrastructure, with critical components receiving additional signing and notarization. ## Stability you can rely on When your business relies on dependable software delivery, you require predictability and consistency: - **Flake stability guarantee**: While flakes remain experimental in upstream Nix, we provide a formal stability guarantee to ensure your workflows remain functional. - **Rigorous release process**: Every change undergoes comprehensive security, performance, and compatibility validation. - **Customer roadmap**: Transparent decision making and predictable release cycles. ## Deployment options for every need From bare metal to virtual machines, cloud to edge, Determinate Nix 3.0 delivers reproducible builds everywhere: - [**First-class macOS support**][nix-darwin-support]: We provide full [MDM] integration, including partnerships with leading MDM providers like [jamf], and ensure seamless macOS upgrades across your Apple ecosystem. - **Performance optimizations**: Ship code faster with improved build times and streamlined workflows. - **Intelligent resource management**: Automated garbage collection monitors system conditions to optimize maintenance. ## Why organizations choose Determinate Nix From financial services to healthcare to critical infrastructure, teams choose Determinate Nix when they require: - Predictable feature implementation with clear timelines for new capabilities. - Security and compliance that meet and exceed industry requirements. - Dramatically faster build times that slash evaluation overhead and accelerate development cycles by 3-4x in typical workloads. - Operational workflows that scale from small projects to global deployments. Determinate Nix 3.0 transforms Nix from a powerful tool into a trusted platform that delivers the governance, performance, and stability that serious production environments demand. ## Get started with Determinate Nix 3.0 today! Ready to experience Determinate Nix 3.0? ### Installing Determinate Nix 3.0 Installation is straightforward whether you're deploying across an organization or installing on your workstation. Our [comprehensive documentation site][determinate] provides tailored installation instructions for most environments. ### Upgrading to Determinate Nix 3.0 Follow these straightforward upgrade instructions for your environment. #### For macOS users Download the universal package and run it. When the installation is complete, open a terminal and run: ```shell title="Check Nix version" nix --version ``` You should see `nix (Determinate Nix 3.0.0) 2.26.3`. #### For NixOS users If you're already using the [Determinate flake][determinate-flake] in your [NixOS] configuration, update the flake: Update the inputs section `flake.nix` to reference the latest Determinate version: ```nix title="flake.nix" { inputs.determinate.url = "https://flakehub.com/f/DeterminateSystems/determinate/*"; } ``` Run a system rebuild to apply the changes: ```shell title="Upgrade NixOS" sudo nixos-rebuild switch --flake . ``` Log in to [FlakeHub]: ```shell title="Log in with determinate-nixd" determinate-nixd login ``` Verify your upgrade: ```shell title="Check Nix version" nix --version ``` You should see `nix (Determinate Nix 3.0.0) 2.26.3`. #### For Linux users First, download the appropriate `determinate-nixd` for your platform: #### Linux (x86_64) ```shell title="Download Determinate Nix for Linux (x86_64)" curl -sSLfo determinate-nixd https://install.determinate.systems/determinate-nixd/stable/x86_64-linux ``` #### Linux (aarch64) ```shell title="Download Determinate Nix for Linux (aarch64)" curl -sSLfo determinate-nixd https://install.determinate.systems/determinate-nixd/stable/aarch64-linux ``` Make the file executable and run the upgrade: ```shell title="Upgrade Determinate Nix" chmod +x ./determinate-nixd sudo ./determinate-nixd upgrade ``` Log in to [FlakeHub]: ```shell title="Log in with determinate-nixd" determinate-nixd login ``` The upgrade process will: - Preserve all your existing Nix configurations - Update to Determinate Nix 3.0 components - Restart the Nix daemon with the new version Verify your upgrade: ```shell title="Check Nix version" nix --version ``` You should see `nix (Determinate Nix 3.0.0) 2.26.3`. You can now remove the temporary installation file: ```shell title="Remove temporary installation file" rm ./determinate-nixd ``` For [FlakeHub] teams, your authentication and access to [private flakes][private-flakes] and [caches][cache] will remain intact throughout the upgrade process. If you encounter any issues during the upgrade or have additional questions, enterprise customers should contact their dedicated account representative directly. All users are also welcome to [join our Discord server][discord] for community support. ## Contact us to transform your Nix experience Ready to transform the way your team builds critical software? Experience Determinate Nix 3.0 today and leave behind dependency nightmares, security headaches, and performance bottlenecks. Email [hello@determinate.systems](mailto:hello@determinate.systems) to discuss how we can empower your team with a robust foundation that enables you to focus on what matters most: creating exceptional software that drives your business forward. [det-nix]: https://docs.determinate.systems/determinate-nix [nix]: https://zero-to-nix.com/concepts/nix [flakehub]: https://flakehub.com [flakes]: https://zero-to-nix.com/concepts/flakes [slsa-3]: https://slsa.dev/spec/v1.0/levels#build-l3 [determinate]: https://docs.determinate.systems [entra-auth]: https://learn.microsoft.com/entra/identity/authentication/overview-authentication [github-auth]: https://docs.github.com/authentication [lazytrees-pr]: https://github.com/NixOS/nix/pull/6530 [parallel-eval-pr]: https://github.com/NixOS/nix/pull/10938 [soc2]: https://trust.determinate.systems/ [zscaler]: https://www.zscaler.com/ [fortinet]: https://www.fortinet.com/ [mdm]: https://docs.determinate.systems/guides/mdm [nix-darwin-support]: /blog/nix-darwin-updates/ [jamf]: https://www.jamf.com/ [determinate-flake]: https://flakehub.com/flake/DeterminateSystems/determinate?view=usage [nixos]: https://zero-to-nix.com/concepts/nixos/ [discord]: https://discord.gg/invite/a4EcQQ8STr [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [cache]: https://docs.determinate.systems/flakehub/cache [pinning]: https://zero-to-nix.com/concepts/pinning --- **Fetch artifacts directly from FlakeHub Cache using `fh fetch`** Published: February 26, 2025 URL: https://determinate.systems/blog/fh-fetch Building stuff with Nix is extremely cool but do you know what's even cooler? Letting another machine build it so that all you have to do is fetch the result. If that sounds appealing then you're in luck because we recently added a `fetch` command to [fh], the CLI for [FlakeHub], that enables you to fetch [closures] directly from [FlakeHub Cache][cache]. Here's an example fetch command that should work on your system if you have [fh] 0.1.22 or later installed: ```shell title="Fetch a package from FlakeHub Cache" NIX_SYSTEM=$(nix eval --impure --expr 'builtins.currentSystem' --raw) fh fetch "DeterminateSystems/flake-iter/*#packages.${NIX_SYSTEM}.default" ./flake-iter ``` This pulls a CLI tool called [flake-iter] directly from FlakeHub Cache, puts it in the [Nix store][store], and writes a symlink to the `./flake-iter` path. With the flake-iter package fetched, you can run the CLI: ```shell title="Run the fetched CLI" ./flake-iter/bin/flake-iter --help ``` No `nix build` command, no [store path evaluation][store-paths-post], just a [flake output reference][flake-ref] and you're all set. ## How to use it In order to use the `fh fetch` command, you need to do a few things: 1. Get started with [Determinate] following the onboarding guide for either [organizations] or [individuals]. This includes [signing up][signup] for a paid plan with FlakeHub, which provides access to FlakeHub Cache. 1. [Install fh][install] version [0.1.22][v0.1.22] or later. 1. Start [pushing things][using] to FlakeHub Cache. 1. [Log in][login] to FlakeHub. ## Example use case: fetching OCI images \{#oci-images} Up at the top, we saw an example of using `fh fetch` to pull a CLI tool from FlakeHub Cache. Another great use case is pulling [Open Container Initiative][oci] (OCI) container images, such as [Docker] or [Podman] images. I'm a macOS-only dev, which means that working with containers in Nix can be challenging because I can't actually *build* Linux containers locally without resorting to a remote builder. I know how to set up a remote builder because I'm a long-time Nix user, but if you're in an organization with many macOS folks, it may be much more straightforward to build images in CI and have people fetch them. Here's an example command that does just that: ```shell title="Fetch an OCI image directly from FlakeHub Cache" fh fetch \ "DeterminateSystems/fh-fetch-example/*#dockerImages.x86_64-linux.server" ./oci-image ``` Since FlakeHub Caches are private, this command *will fail* for anyone outside of the `DeterminateSystems` org. Fetching only pulls from FlakeHub Cache—it never builds or even evaluates the store path—and without access to that specific flake, you're out of luck. And so even though both the [repo][example-repo] and the [flake on FlakeHub][example-flake] are public, the fetchable output is not (it would also be inaccessible, of course, if the repo or flake were [private][private-flakes]). But since *I* am allowed to fetch, the command worked and I can now load the image into [Docker]: ```shell title="Load the Docker image from the target link" docker load < ./oci-image ``` That provides this output: ```shell title="Log output with the container's name and tag" Loaded image: web-server:5mll9s6m7m7pjha0k2krzq8kvjn111i3 ``` And now I can run the image: ```shell title="Run the recently loaded image" docker run \ --publish 8080:8080 \ web-server:lpr4bbf6v5yfzn1v1cfvlr69vbcr0iqh ``` Success! Now I as a macOS user have a brand new option for working with Nix-built OCI images. This is one use case among many for `fh fetch`, so be on the lookout for Nix-build things that you may need to securely distribute inside your own org. ## The FlakeHub platform branches out With `fh fetch`, FlakeHub is beginning to transform into a powerful platform not just for secure access to [flakes] and secure binary caching but also as a **distribution mechanism** for things that you've built using Nix. The ability to build and cache in CI—potentially from [private flakes][private-flakes]—and then provide your teams with secure access to built [closures] is genuinely novel in the Nix ecosystem. How you use these distribution channels as a building block inside your organization is up to you, of course, but we're excited to see the directions that you take these novel capabilities. [cache]: https://docs.determinate.systems/flakehub/cache [closures]: https://zero-to-nix.com/concepts/closures [determinate]: https://docs.determinate.systems [docker]: https://docker.com [example-flake]: https://flakehub.com/flake/DeterminateSystems/fh-fetch-example [example-repo]: https://github.com/DeterminateSystems/fh-fetch-example [fh]: https://flakehub.com/flake/DeterminateSystems/fh [flake-iter]: https://github.com/DeterminateSystems/flake-iter [flake-ref]: https://zero-to-nix.com/concepts/flakes/#references [flakehub]: https://flakehub.com [flakes]: https://zero-to-nix.com/concepts/flakes [individuals]: https://docs.determinate.systems/getting-started/individuals [install]: https://docs.determinate.systems/flakehub/cli/#installation [login]: https://docs.determinate.systems/flakehub/cli/#login [oci]: https://opencontainers.org [organizations]: https://docs.determinate.systems/getting-started/organizations [podman]: https://podman.io [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [signup]: https://flakehub.com/signup [store]: https://zero-to-nix.com/concepts/nix-store [store-paths-post]: /blog/resolved-store-paths [using]: https://docs.determinate.systems/flakehub/cache/#using-flakehub-cache [v0.1.22]: https://flakehub.com/flake/DeterminateSystems/fh/0.1.22 --- **Introducing Determinate AMIs for NixOS** Published: February 24, 2025 URL: https://determinate.systems/blog/nixos-amis Today, we're excited to announce that Determinate Systems now offers [Amazon Machine Images][amis] (AMIs) for [NixOS] that include [Determinate Nix][det-nix]. Our AMIs are available for both AMD64 Linux (`x86_64-linux` in Nix terms) and ARM64 Linux (`aarch64-linux`) and you can see the code behind them in our [nixos-amis] repo. What sets these apart from other NixOS AMIs is that they seamlessly interoperate with [FlakeHub] and thus offer much speedier NixOS deployments, even inside of [autoscaling groups][autoscaling]. Interoperation with FlakeHub is provided by [Determinate Nix][det-nix] and [fh]: * [Determinate Nix][det-nix], our validated and secure [Nix] for enterprises. Determinate Nix includes [Determinate Nixd][dnixd], a utility that enables you to log in to [FlakeHub] using [AWS Secure Token Service][sts] (STS) using one command: ```shell title="Log in to FlakeHub from AWS with a single command" determinate-nixd login aws ``` * [fh], the CLI for [FlakeHub]. You can use fh for things like [applying][fh-apply] NixOS configurations uploaded to [FlakeHub Cache][cache]. Here's an example: ```shell title="Apply a NixOS configuration to your AMI with two commands" determinate-nixd login aws fh apply nixos \ "my-org/my-flake/*#nixosConfigurations.my-nixos-configuration" ``` In this case, fh would pull the target NixOS [closure] from FlakeHub Cache without even needing to [evaluate the store path][store-paths]. We see this as a major upgrade over existing approaches to NixOS deployment, and you can read more about it in a [recent blog post][resolved-paths]. ## Using Terraform or OpenTofu While there are lots of ways to deploy AMIs to [EC2], we suspect that users are likely to gravitate toward [Infrastructure as Code][iac] (IAC) tools like [Terraform] and [OpenTofu]. Here's an example declaration of an [`aws_ami`][aws-ami-resource] data resource that uses our AMD64 Linux AMI: ```hcl title="Terraform/OpenTofu configuration" data "aws_ami" "detsys_nixos" { most_recent = true # Determinate Systems' AMI owner ID owners = ["535002876703"] # Our AMIs are currently under the epoch-1 version filter { name = "name" values = ["determinate/nixos/epoch-1/*"] } # AMD64 Linux filter { name = "architecture" values = ["x86_64"] } } ``` With that AMI declared, you can create an [EC2] instance that applies a [NixOS] configuration upon launch: ```hcl title="An EC2 instance with an applied NixOS configuration" {2,6-7} resource "aws_instance" "ethercalc-server" { ami = data.aws_ami.detsys_nixos.id instance_type = "t3.micro" user_data = < ## What you should change If you're currently using Determinate but not nix-darwin, you can start using nix-darwin at any time. Just make sure to set `nix.enable = false` at the top level of your nix-darwin configuration. If you're currently using Determinate with the [`determinate` module][determinate-flake] for nix-darwin, follow these steps: - Upgrade both the `determinate` and the `nix-darwin` input in your configuration using `nix flake update`. - Run `darwin-rebuild` for your configuration. This should *fail*. - After that fails, set `nix.enable = false;` and remove the Determinate nix-darwin module from your configuration. - If you have Nix configuration you're currently setting with nix-darwin, add that to the `/etc/nix/nix.custom.conf` configuration file instead. And that's it! Nix is now managed by Determinate, while nix-darwin handles your macOS system configuration. In terms of nix-darwin release compatibility, this approach works on both the 24.11 release branch as well as the rolling `master` branch. If you're not yet using Determinate on macOS, you can get started now: ## Implications Letting Determinate manage Nix while letting nix-darwin manage macOS system configuration plays much better to the strengths of both tools. And beyond the benefit to Determinate users, providing nix-darwin users with an opt-out switch like this means that other tools and platforms are also free to manage Nix on macOS. More configurability, after all, is the Nix way. [announcement]: /blog/announcing-determinate-nix [determinate]: https://docs.determinate.systems [determinate-flake]: https://github.com/DeterminateSystems/determinate [emily]: https://github.com/emilazy [gui]: /blog/graphical-nix-installer [macos]: /nix/macos/overview [nix-daemon]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-daemon [nix-enable-opt]: https://daiderd.com/nix-darwin/manual/index.html#opt-nix.enable [nix-darwin]: https://github.com/LnL7/nix-darwin [pr-1313]: https://github.com/LnL7/nix-darwin/pull/1313 [pr-1326]: https://github.com/LnL7/nix-darwin/pull/1326 [survival-mode]: /blog/nix-survival-mode-on-macos --- **Changelog: improved nix-darwin support for Determinate!** Published: February 18, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-033 We're excited to announce a handful of quality-of-life improvements to [Determinate]: * Determinate now ships with [Nix 2.26.2][2.26.2]. * [FlakeHub Cache][cache] is no longer added to your [`nix.conf`][nix.conf] substituters until you're logged in, eliminating the confusing `not a binary cache` error. * Much improved [nix-darwin] support! People who use both [Determinate] and nix-darwin should now set `nix.enable = false` in their nix-darwin configuration. This enables Determinate to handle Nix installation and configuration rather than nix-darwin, as was the case before. Many gracious thanks to [Emily] on the nix-darwin team for helping us with this. We'll have much more to say about this soon. This release also has two bug fixes specifically targeted to enterprise users: * Enterprise [macOS] users with [Time Machine][time-machine] disabled by [Mobile Device Management][mdm] (MDM) can now install Determinate. * Users installing Nix from the Determinate macOS package are now notified in more cases if MDM prevents mounting volumes. And finally, we also shipped a number of reliability improvements for the Determinate [installer]: * We can now tolerate and mount the Nix Store APFS volume if it's unlocked but not yet mounted. * Uninstalling Determinate now cleans up the generated `nix.conf` file. * Invalid or corrupted `nix.conf` files no longer block installation or upgrades. * Creating `/etc/profiles.d/nix.sh` now creates `/etc/profiles.d` if it doesn't exist already. * The installation process is now far more resilient to strange behavior on XFS while unpacking the Nix tarball. To [upgrade] Determinate now: ```shell title="Upgrade Determinate Nix" sudo determinate-nixd upgrade ``` If you aren't using Determinate yet, check out our getting started guides for [organizations] and [individuals]. [2.26.2]: https://discourse.nixos.org/t/nix-2-26-released/59211 [cache]: https://flakehub.com/cache [determinate]: https://docs.determinate.systems [emily]: https://github.com/emilazy [individuals]: https://docs.determinate.systems/getting-started/individuals [installer]: https://github.com/DeterminateSystems/nix-installer [macos]: /blog/tags/macos [mdm]: https://docs.determinate.systems/guides/mdm [nix.conf]: https://flakehub.com/nix-conf [nix-darwin]: https://github.com/LnL7/nix-darwin [organizations]: https://docs.determinate.systems/getting-started/organizations [time-machine]: https://support.apple.com/104984 [upgrade]: https://docs.determinate.systems/determinate-nix#determinate-nixd-upgrade --- **Improved evaluation times with pre-resolved Nix store paths** Published: February 12, 2025 URL: https://determinate.systems/blog/resolved-store-paths Building things with Nix is pretty amazing. You point Nix at a flake reference for a derivation and say "build!" It then spins up a sandbox and [reproducibly][reproducibility] creates real bytes on disk. Those bytes on disk could be a single `hello.txt` file or a hefty Docker image or a fully functional [NixOS] system—or much else besides. But the power of `nix build` comes at a cost, and not just in terms of disk space. Before it writes anything to disk, Nix first needs to evaluate the [store path][paths] of the derivation that it's going to [realise][realisation]. It then uses that store path to determine whether it can pull the path from a [cache][caching] like [FlakeHub Cache][flakehub-cache] or whether it needs to build it instead. Finally, it applies this logic recursively throughout the entire dependency tree of the derivation and you end up with the desired bytes on disk. ## The evaluation tax Even if you can (ideally) pull a path from a cache instead of building it, you still have to pay Nix's *evaluation tax* in determining the derivation's store path in the first place. This cost is no big deal if you're on a modern workstation or a beefy build server. But on resource-constrained devices, such as smart sensors, Raspberry Pis, or industrial controllers, even pulling from a cache can use a ton of memory if you need to evaluate the store path—something you'd surely like to avoid at all costs. ## The solution: resolved store paths \{#solution} Fortunately, [FlakeHub] now offers an elegant solution to this problem: [**resolved store paths**][docs]. With this feature, you can do two things that have not yet been possible in the Nix ecosystem: 1. You can provide FlakeHub with a [flake output reference][flakeref] and get the store path for that reference *without using Nix*. With the store path in hand, you can pull directly from [FlakeHub Cache][flakehub-cache] *with no evaluation tax*. 1. You can apply a [NixOS], [Home Manager][hm], or [nix-darwin] configuration to the current host solely with a flake reference—again pulled directly from FlakeHub Cache without incurring an evaluation tax. Resolving flake outputs is useful only in conjunction with [FlakeHub Cache][flakehub-cache], which is available if you [sign up][signup] for a [FlakeHub] paid plan. Paid plans also provide access to features like [private flakes][private-flakes]. ## Working with resolved store paths You can work with resolved store paths using [fh], the CLI for FlakeHub, which enables you to [fetch store paths](#fh-resolve) and [apply configurations from flake references](#fh-apply). ## `fh resolve` \{#fh-resolve} [fh], the CLI for FlakeHub, has a [`resolve`][fh-resolve] command that resolves [flake references][flakeref] to store paths by fetching them directly from FlakeHub. Here's an example: ```shell title="Resolving the path to a NixOS configuration" fh resolve \ "DeterminateSystems/store-paths/*#nixosConfigurations.baseline" ``` It also works with the full flake reference: ```shell title="Resolving the path to a NixOS configuration with a full flake reference" fh resolve \ "https://flakehub.com/f/DeterminateSystems/store-paths/*#nixosConfigurations.baseline" ``` Both of these fh commands return this store path immediately: ``` /nix/store/84g3x33v5srlljhfaskwy7mmar5697vn-nixos-system-minimal-24.05.20240714.53e81e7 ``` It may not be clear why this is special, as you *can* get the same result using plain old [`nix eval`][nix-eval]: ```shell title="Get store path info" nix eval --raw "https://flakehub.com/f/DeterminateSystems/garage-door/0.1.9#nixosConfigurations.turner.config.system.build.toplevel" ``` But this command requires Nix to be available and takes several seconds and a fair amount of memory and disk space to complete. With `fh resolve`, by contrast, fh fetches an *already calculated store path* stored in FlakeHub's database. In a variety of deployment scenarios, using fh is a vastly superior option. ## `fh apply` \{#fh-apply} [fh], the CLI for FlakeHub, has an [`apply`][fh-apply] command that you can use to apply [NixOS](#nixos), [Home Manager](#home-manager), and [nix-darwin](#nix-darwin) configurations to the current system. `fh apply` is powered by the same path resolution feature mentioned above, meaning that you can use it to, for example, apply a [NixOS](#nixos) configuration to the current host. ### NixOS Let's say that you have a [NixOS] configuration as a [flake output][output] on your `my-org/nixos-configs` repo, while that configuration is being pushed to [FlakeHub Cache][flakehub-cache]. With [fh], you could apply that configuration using this single command: ```shell title="Apply a NixOS configuration from a flake output reference" fh apply nixos "my-org/nixos-configs/0.1#nixosConfigurations.staging-server" ``` If you leave off the flake output reference—`nixosConfigurations.staging-server` in this case—fh "magically" infers that you want the `nixosConfigurations.$(hostname)` output. If that's the case, you can shorten the reference to this: ```shell title="Apply a NixOS configuration from a flake output reference" fh apply nixos "my-org/nixos-configs/0.1" ``` ### NixOS on Amazon Web Services (AWS) \{#nixos-aws} The value of this new approach becomes clear in this little snippet of code: ```shell title="Log in to AWS and apply a NixOS configuration" determinate-nixd login aws fh apply nixos "my-org/nixos-configs/0.1" ``` If you have an [EC2] instance with [Determinate Nix][det-nix] installed, this is all you need to run to update your NixOS system configuration. [Determinate Nixd][dnixd] logs you in, which enables you to use [private flakes][private-flakes], and then `fh` does its thing. As before, it does so without needing to evaluate the store path, which makes this speedier than your typical NixOS deployment. And because FlakeHub supports [semantic versioning][semver] (SemVer) for flakes, the `fh apply` operation would automatically use the most recent release within the supplied version constraint, so you could also use constraints like `*` (whatever is latest), `=0.1.2` (exact match), or `~1.2` (flexible patch). ### Home Manager While `fh apply` for NixOS works swimmingly in cloud environments like AWS, it also supports applying [Home Manager][hm] configurations. Here's an example: ```shell title="Apply a Home Manager configuration from a flake output reference" fh apply home-manager "my-org/home-configs/0.1#homeConfigurations.justme" ``` `fh apply` has a default flake output name of `homeConfigurations.$(whoami)`, so this command without the flake output would work if the output name conformed to that: ```shell title="Apply a NixOS configuration from a simplified reference" fh apply home-manager "my-org/home-configs/0.1" ``` ### nix-darwin And finally, `fh apply` also supports [nix-darwin]. We at [Determinate Systems][detsys] are committed to providing first-class Nix support on macOS and this is a testament to that. Here's an example command: ```shell title="Apply a nix-darwin configuration from a flake output reference" fh apply nix-darwin "my-org/macos-configs/0.1#darwinConfigurations.justme-aarch64-darwin.system" ``` The default flake output name for nix-darwin is `darwinConfigurations.${hostname}`, where `hostname` is the value of `scutil --get LocalHostName`. If your flake output name conforms to that, you can run this: ```shell title="Apply a nix-darwin configuration from a simplified reference" fh apply nix-darwin "my-org/macos-configs/0.1" ``` ## Generating store paths Above, we saw how to use [`fh resolve`](#fh-resolve) and [`fh apply`](#fh-apply) with resolved store paths. To be able to fetch those paths, however, you need to generate them as part of the [flake publishing process][publishing]. You can configure that with a one-liner in the [flakehub-push] Action: ```yaml title=".github/workflows/flakehub.yml" {4} - uses: DeterminateSystems/flakehub-push@main with: name: my-org/my-flake include-output-paths: true ``` With this configuration in place, the store paths for all of your flake's outputs are evaluated and stored in FlakeHub's database. As with everything on FlakeHub, those paths are available only to users with proper permissions. For [public flakes][public-flakes], that's anyone with [fh] installed; for [private flakes][private-flakes] that's only those to whom you've explicitly granted access. Here's a full GitHub Actions workflow with resolved store paths enabled: ```yaml title=".github/workflows/flakehub.yml" {23} name: Publish flake with resolved store paths available on: push: branches: - main jobs: flakehub-publish: runs-on: ubuntu-22.04 needs: build-docker-image permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/flakehub-push@main with: name: omnicorp/web rolling: true # Publish on every push to the default branch visibility: private include-output-paths: true ``` With that output path published, users with access to the `omnicorp/web` flake could use fh to fetch a resolved path like this: ```shell title="Resolve store path after publishing" fh resolve "omnicorp/web/*#dockerImages.x86_64-linux.server" ``` ## Truly new horizons for Nix [FlakeHub] now enables you to determine store paths without evaluating them, which unlocks some exciting new possibilities for Nix in the enterprise. With this feature, [`fh apply`](#fh-apply) can apply Nix-based configurations for [NixOS](#nixos), [Home Manager](#home-manager), and [nix-darwin](#nix-darwin) in a much more efficient way than is currently available in the Nix ecosystem. Beyond `fh apply`, we can imagine people using [`fh resolve`](#fh-resolve) for things like scripting or pulling artifacts like Docker containers from [FlakeHub Cache][cache] (more on this soon). [cache]: https://flakehub.com/cache [caching]: https://zero-to-nix.com/concepts/caching [det-nix]: https://docs.determinate.systems/determinate-nix [detsys]: https://determinate.systems [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [docs]: https://docs.determinate.systems/flakehub/store-paths [ec2]: https://aws.amazon.com/ec2 [fh]: https://docs.determinate.systems/flakehub/cli [fh-apply]: https://docs.determinate.systems/flakehub/cli#apply [fh-resolve]: https://docs.determinate.systems/flakehub/cli#resolve [flakehub]: https://flakehub.com [flakehub-cache]: https://docs.determinate.systems/flakehub/cache [flakehub-push]: https://github.com/DeterminateSystems/flakehub-push [flakeref]: https://zero-to-nix.com/concepts/flakes/#references [hm]: https://flakehub.com/flake/nix-community/home-manager [nix-darwin]: https://github.com/LnL7/nix-darwin [nix-eval]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-eval [nixos]: https://zero-to-nix.com/concepts/nixos [output]: https://zero-to-nix.com/concepts/flakes#outputs [paths]: https://zero-to-nix.com/concepts/nix-store/#store-paths [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [public-flakes]: https://docs.determinate.systems/flakehub/concepts/visibility#public [publishing]: https://docs.determinate.systems/flakehub/publishing [realisation]: https://zero-to-nix.com/concepts/realisation [reproducibility]: https://zero-to-nix.com/concepts/reproducibility [semver]: https://docs.determinate.systems/flakehub/concepts/semver [signup]: https://flakehub.com/signup --- **Supercharging Nix deployments with FlakeHub Cache** Published: January 24, 2025 URL: https://determinate.systems/blog/home-manager-deployments-with-fh Today, I'm excited to share how [FlakeHub Cache][cache] can dramatically improve your Nix development *and* deployment workflow. While our [Magic Nix Cache][mnc] has helped teams boost CI performance over the past year, FlakeHub Cache represents the next evolution - accelerating builds and fundamentally transforming how Nix deployments work. We'll examine a [Home Manager][hm] configuration that demonstrates how FlakeHub Cache's pre-built [closures] and pre-evaluated store paths can make deployments both faster and more reliable, especially in resource-constrained environments. If you're currently using Magic Nix Cache, which will [reach end-of-life for its free tier][mnc-eol] on **February 1st, 2025**, this is the perfect time to explore how FlakeHub Cache can enhance your entire Nix workflow. ## The challenge with traditional Nix deployments If you've worked with Nix, you're likely familiar with the standard deployment flow: evaluate Nix expressions, [realise][realisation] closures, and download or build missing dependencies. While this works, it can be time-consuming and resource-intensive, especially when deploying to multiple machines or CI environments. ## Enter FlakeHub Cache FlakeHub Cache fundamentally changes how Nix deployments work by eliminating build time and evaluation overhead. Here's why this matters: Traditional Nix deployments, even with a binary cache, require two steps that can be resource-intensive: - **Evaluation**: Nix must evaluate your configuration to determine the store paths it needs - this alone can overwhelm memory-constrained devices - **Realization**: Nix then either builds those paths or downloads them from a binary cache FlakeHub Cache eliminates both steps. When you push configurations through CI, FlakeHub: - Evaluates and stores the exact store paths your configuration needs - Caches the built closures for those paths Using FlakeHub Cache when deploying means you can bypass Nix evaluation entirely: ```shell title="Apply the Home Manager configuration supplying only a flake output" fh apply home-manager "determinatesystems/home-manager-example/*#homeConfigurations.linuxUsername@linuxHostname" ``` This single command is transformative - especially for resource-constrained environments where evaluating a Nix configuration could cause out-of-memory errors. FlakeHub already knows the store paths needed, so no local Nix evaluation is required. It fetches the pre-built closure and activates it. ## Real benefits for real teams What makes this approach particularly powerful? - **Zero evaluation overhead**: Skip not just building but even the memory-intensive process of evaluating Nix expressions - **Resource efficient**: Deploy to severely constrained environments that couldn't even evaluate complex Nix configurations - **Consistent environments**: Every machine gets bit-for-bit identical outputs - **CI verified**: Configurations are pre-tested before reaching production - **Bandwidth optimized**: Only download what you need ## See it in action We've created a complete example repository that demonstrates these concepts. The configuration supports both macOS and Ubuntu systems, showing how to: - Structure your flake for multi-platform support - Configure GitHub Actions to build and cache your configurations - Deploy configurations rapidly using `fh` Check out the [complete example on GitHub][hm-example] to see how it all fits together. ## Beyond Home Manager While this example focuses on Home Manager, the same principles apply to: - NixOS system configurations - nix-darwin configurations - Development environments - Custom tooling and packages The benefits become even more pronounced with larger configurations that traditionally require significant build time. ## Getting started Ready to supercharge your own Nix deployments? Here's how to get started: 1. [Sign up for FlakeHub][signup] to get access to FlakeHub Cache. 2. Get one month free by applying the coupon code **FHC** during checkout. If you're a maintainer of an open source project, contact us at `support@flakehub.com` for a free account. 3. Review our [comprehensive documentation][flakehub-docs] to learn more about FlakeHub's capabilities. 4. Follow our [getting started guide][getting-started] to begin your journey. ## Join the revolution FlakeHub Cache represents a fundamental shift in how we think about Nix deployments. By moving computation from deployment time to build time, we're making Nix more accessible and practical for teams of all sizes. Have questions or want to learn more? Join us on [Discord]. We'd love to hear about your use cases and help you get started with FlakeHub Cache. Remember: your deployments should be boring and predictable, providing a sense of security and stability. With FlakeHub Cache, they can be. [cache]: https://docs.determinate.systems/flakehub/cache [closures]: https://zero-to-nix.com/concepts/closures [discord]: https://determinate.systems/discord [flakehub-docs]: https://docs.determinate.systems/flakehub [getting-started]: https://docs.determinate.systems/getting-started [hm]: https://github.com/nix-community/home-manager [hm-example]: https://github.com/DeterminateSystems/home-manager-example [realisation]: https://zero-to-nix.com/concepts/realisation [signup]: https://flakehub.com/signup [mnc]: https://github.com/DeterminateSystems/magic-nix-cache [mnc-eol]: /blog/magic-nix-cache-free-tier-eol --- **End of life for the free tier of the Magic Nix Cache** Published: January 21, 2025 URL: https://determinate.systems/blog/magic-nix-cache-free-tier-eol We recently became aware that the API backing [Magic Nix Cache][mnc]'s free CI cache in [GitHub Actions][actions] will be shut down on February 1st, 2025. This will require most users to change their [GitHub Actions][actions] workflows. The TL;DR: * The free Magic Nix Cache will stop working unless you're on [GitHub Enterprise Server][gh-enterprise]. * Users can upgrade to [FlakeHub Cache][flakehub-cache] and get **one month free** using the coupon code `FHC`. * Open source projects can request a free account at [support@flakehub.com][email]. ## Background When we set out to build Magic Nix Cache, we studied the code of the [GitHub Actions Cache][gha-cache] and found that the API for the backend was relatively trivial, consisting of just a few standard HTTP calls. With the help of [Attic]'s creator, [Zhaofeng Li][zhaofeng], we created Magic Nix Cache based on that API. Last September, GitHub published a [blog post][gha-post] about shutting down that API. On December 5th, 2024, GitHub merged and released an updated Cache Action using the [new API][new-api-pr]. Unfortunately, we didn't see either blog post or the updated API until last week, which gives us limited time to respond. ## Analysis Some information we learned when researching this new API: * GitHub's new cache API is built on [Twirp], a [Protocol-Buffers][protobuf]-based RPC framework built by [Twitch]. * Their updated client code became available first in September and was merged in late November. * The new client code is generated from a Protocol Buffers `.proto` file, but the original source `.proto` file has [not been made available][proto-file-issue]. Before we released Magic Nix Cache, we contacted GitHub in every way we could to see if it was acceptable. We never heard back, assumed that no news is good news, and moved forward. Had GitHub released their `.proto` files, we would have a clear path forward—but they didn't. Whether or not GitHub is even *aware* of Magic Nix Cache, I believe that GitHub is sending a clear message that they don't want tools like Magic Nix Cache to integrate with their cache the way that we have. The only options we have at this point are: * Rewrite Magic Nix Cache from scratch in [TypeScript] and use the [Actions Toolkit][gh-sdk]. * Reverse engineer the `.proto` source to generate a [Rust] client. * Discontinue Magic Nix Cache's free GitHub-Actions-based cache. One of the risks of repurposing an API in an unexpected way is that it can change and break on short notice. And because it *is* such short notice, I don't see the first two options working out in the timeline available to us. ## The future I hope that we can bring back Magic Nix Cache's free cache for GitHub Actions. It was the most frictionless way to get a free binary cache to make Nix CI faster. This is not what I hoped for, and I would be happy to see GitHub release their `.proto` files, which would enable us to update Magic Nix Cache to use it. In the future, we may also reimplement it in TypeScript using their official [Actions Toolkit][gh-sdk]. ## The present We are not able to migrate Magic Nix Cache to use their new APIs in the timeline available. Because of that, users of the free version of Magic Nix Cache will start seeing their CI runs fail as soon as GitHub sunsets their v1 cache API. Any builds still using the [Magic Nix Cache Action][mnca] will begin to **fail** once GitHub proceeds with decommissioning the v1 cache API. We have opted to fail builds rather than allowing them to proceed quietly without using the cache because we want to prevent those builds from silently using costly build minutes. [GitHub Enterprise Server][gh-enterprise] users can continue to use Magic Nix Cache as before, as GitHub Enterprise Server still has the old v1 cache API available, for now. If you'd like to continue using a Nix [binary cache][cache] in GitHub Actions, you can switch to [FlakeHub Cache][flakehub-cache] and get one month free by applying the coupon code `FHC` during checkout. If you're a maintainer of an open source project, contact us at [support@flakehub.com][email] for a free account. [actions]: https://docs.github.com/actions [attic]: https://github.com/zhaofengli/attic [cache]: https://zero-to-nix.com/concepts/caching [email]: mailto:support@flakehub.com [flakehub-cache]: https://flakehub.com/cache [gh-enterprise]: https://github.com/enterprise [gha-cache]: https://github.com/actions/cache [gha-post]: https://github.blog/changelog/2024-09-16-notice-of-upcoming-deprecations-and-changes-in-github-actions-services [gh-sdk]: https://github.com/actions/toolkit [mnc]: https://github.com/determinatesystems/magic-nix-cache [mnca]: https://github.com/determinatesystems/magic-nix-cache-action [new-api-pr]: https://github.com/actions/cache/pull/1509 [proto-file-issue]: https://github.com/actions/toolkit/issues/1931 [protobuf]: https://protobuf.dev [rust]: https://rust-lang.org [twirp]: https://github.com/twitchtv/twirp [twitch]: https://twitch.tv [typescript]: https://typescriptlang.org [zhaofeng]: https://github.com/zhaofengli --- **Changelog: customer binding, nix.conf, and yanked Nix 2.25.4** Published: January 14, 2025 URL: https://determinate.systems/blog/changelog-determinate-nix-030 We'd like to announce some noteworthy changes to [Determinate Nix][det-nix] involving [customer binding](#binding), [user customization](#customization), [Determinate in OCI containers](#containers), and a [yanked Nix release](#yanked-nix-release). We've also cut two new releases for the [Determinate Nix Installer](#installer-releases). ### Customer binding \{#binding} We've updated [Determinate Nixd][dnixd] to enable users to bind a [Determinate] installation to a specific [FlakeHub] organization. When you do this, it prevents users from logging in to a FlakeHub account that isn't associated with the organization. Here's an example binding command: ```shell title="Binding a Determinate installation to a specific org" determinate-nixd auth bind AmbiguousTechnologies ``` For persistent CI workers, we recommend pre-binding the [Determinate] installation to prevent CI jobs from substituting from outside caches. For more information, see our documentation on [binding your installation][binding]. ### Splitting user customization from Determinate defaults in the Determinate Nix Installer \{#customization} The [Determinate Nix Installer][installer] now writes user-specified configuration entries to a `nix.custom.conf` file. The default Nix configuration is written to `nix.conf`, which imports the `nix.custom.conf`. We made this change in [collaboration with Emily from the nix-darwin project][nix-darwin-collab], for which we are grateful. At the same time, Determinate now takes a more prescriptive approach to the `nix.conf`. Previously, Determinate left the `nix.conf` alone if it was a symlink. This created unpredictable behavior for some users. Determinate will now move the `nix.conf` out of the way to a backup file and write its own at startup. User-specified settings can still be saved to `nix.custom.conf`. Determinate also previously tried to migrate the contents of `nix.conf` to the custom file, but the migration was unacceptably sloppy and error prone. Users reported seeing their `nix.custom.conf` growing with time, which is certainly not ideal. Consider reviewing your custom configuration and deleting any duplicate entries. ### Determinate in OCI containers \{#containers} While installing upstream Nix with the [Determinate Nix Installer][installer] generally works quite well in [Open Container Initiative][oci] (OCI) containers, installing [Determinate] has generally not worked properly and we are not currently testing for container compatibility. If your use case involves Determinate in OCI containers, [reach out][email] to us and we can put improved container support on our roadmap. ### Nix 2.25.4 yanked from DeterminateSystems/nix on FlakeHub \{#yanked-nix-release} During our release process for Nix 2.25.4, we saw a [regression] in Nix around relative `path:.` flake references. Nix 2.25.4 was briefly made available through our [`nix`][nix] and [`determinate`][determinate] flakes. Those releases have been [yanked][yanked-releases] and rolled back to [2.25.3][non-yanked-version]. We [submitted a patch][patch] for the bug and will validate and roll out Nix 2.25.5 as soon as possible. ## Determinate Nix Installer releases \{#installer-releases} In addition to changes to Determinate Nix, we recommend checking out these recent releases of the Determinate Nix Installer: * [v0.33.0](https://github.com/DeterminateSystems/nix-installer/releases/tag/v0.33.0) * [v0.32.3](https://github.com/DeterminateSystems/nix-installer/releases/tag/v0.32.3) [binding]: https://docs.determinate.systems/determinate-nix#determinate-nixd-binding [determinate]: https://flakehub.com/flake/DeterminateSystems/determinate [det-nix]: https://docs.determinate.systems/determinate-nix [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [email]: mailto:hello@determinate.systems [flakehub]: https://flakehub.com [installer]: https://github.com/DeterminateSystems/nix-installer [nix]: https://flakehub.com/flake/DeterminateSystems/nix [nix-darwin-collab]: https://github.com/LnL7/nix-darwin/pull/1266 [non-yanked-version]: https://flakehub.com/flake/DeterminateSystems/nix/2.25.3 [oci]: https://opencontainers.org [patch]: https://github.com/NixOS/nix/pull/12254 [regression]: https://github.com/NixOS/nix/issues/12248 [yanked-releases]: https://docs.determinate.systems/flakehub/concepts/yanked-releases --- **Determinate Nix now supports migrating from other caches** Published: December 18, 2024 URL: https://determinate.systems/blog/flakehub-cache-migration In October, we [released Determinate Nix][announcing-determinate-nix] and announced that [private flakes and FlakeHub Cache][ga-flakehub] are now generally available. FlakeHub authentication is often automatic, using cloud native mechanisms like [JSON Web Tokens][jwt] (JWTs) and [Amazon Instance Metadata Service][imds] (IMDS), eliminating from your workflows cumbersome tasks like sharing credentials and manually rotating tokens. As a stark alternative, Determinate Nix manages the lifecycle of these tokens by automatically synthesizing a [netrc] file that Nix can use to authenticate with FlakeHub. That means that your entire organization has a frictionless path to using features like [FlakeHub Cache][cache] and [private flakes][private-flakes]. Since that initial release, customers migrating to FlakeHub Cache have indicated that they want to read from their old cache during the transition. These customers are excited by FlakeHub Cache's support for [Single sign-on][sso] (SSO) and rotating per-user credentials but they still have static credentials to their fleet of machines shared amongst their teams. To serve these customers—any another potential FlakeHub users—we now support extending our synthesized netrc file with customer-defined entries, and we recently published a new guide for [Migrating to FlakeHub Cache][migrating]. Sign up now at https://flakehub.com/signup and visit our [getting started docs][getting-started] to get the best Nix workflow for your team. And as always, we'd love to hear your feedback on our Discord at https://determinate.systems/discord. [announcing-determinate-nix]: /blog/announcing-determinate-nix [cache]: https://docs.determinate.systems/flakehub/cache [ga-flakehub]: /blog/flakehub-cache-and-private-flakes/ [getting-started]: https://docs.determinate.systems/getting-started [imds]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html [jwt]: https://jwt.io [migrating]: https://docs.determinate.systems/guides/migrate-to-flakehub-cache [netrc]: https://everything.curl.dev/usingcurl/netrc.html [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [sso]: https://en.wikipedia.org/wiki/Single_sign-on --- **The future of software is Nix** Published: October 25, 2024 URL: https://determinate.systems/blog/the-future-is-nix The future of software is [Nix][z2n-nix] and we at Determinate Systems want to have a role in building that future. Today, I want to talk about [Determinate], and I want to speak to you from a more personal place than you might be accustomed to here on our blog. ## Determinate, to live up to the promise of our industry What if every machine had a modern Nix with [flakes enabled][nix-installer] out of the box? And the [cache] just worked? And [private repositories][private-flakes] were straightforward to use? And Nix was up to date? And updating your dependencies took little to no effort? And the Nix store didn't occasionally take up your entire disk, with no clear path to solving that? What if adopting Nix were straightforward? What if adopting Nix didn't involve solving a zillion annoying puzzles? What if adopting Nix didn't irritate your IT or security teams? That's our vision. To make a smooth path from "Nix could solve my dev, build, and configuration management problems" to "Nix _is_ solving these problems" and in a way that doesn't involve turning someone from a developer to "the Nix person." For users to pick up Nix and be effective and deploy critical software and not have to spend every waking moment solving unrelated puzzles. For users to have components that fit together in straightforward ways to let them actually do the thing that is important—where the important thing is *not* toying with Nix. A future where software is maintainable and secure and we can take big leaps and still be safe and confident it'll be fine. Where we can take risks and undo it. Where folks building critical infrastructure or deploying to space feel confident choosing Nix. Where Nix is mature enough for their use case. Where security teams know where to find clearly communicated, timely, advisories? That's why we're here. To make more Nix users. To make those users happier and more productive. To make critical infrastructure safer. ## How I got here I knew Nix was the future in 2016 when I found Nix and switched all my personal systems and work infrastructure to it within just a few weeks. And I know this now more than ever. I can feel it in my bones. ## Immutable infrastructure is the way By 2014 I had been burned by declarative configuration management tools like [Chef] and [Puppet]. That experience spurred me to convince the startup where I worked to aggressively adopt immutable infrastructure. With Chef, I had experienced severe drift not just from unmanaged config changes but also because the world around us was changing. I'd come in the next day, push out a change with Chef, and the unmanaged configuration met the new managed configuration and kaboom: production outage. I'd also been burned by the [Docker]'s promise of reproducibility. The company where I worked at the time was in the process of aggressively adopting Docker for all of its deployments. Every merge to one of our dozens of repositories created fresh Docker images. We pushed those images to a registry and then kicked off a CI/CD pipeline that would [use Packer to build a brand new AMI][packer-ami]. We'd then do a blue/green-style rolling deployment to our autoscaling groups to finalize the deploy. It was brilliant. I started talking at conferences about the importance of immutable infrastructure and about how incredibly stable and resilient our deployments and systems were. ### Chef and Docker weren't enough for great immutable infrastructure But our CI/CD pipeline was slow. And it broke—a lot. We learned two things: 1. Trying to manage individual packages or general system state with [Chef] was really hard. An annoying but solvable part of this came from resources like users where, if you added a user then wanted to remove them, you had to explicitly write that "remove" code. The bigger issue was that steps like "install nginx" could do different things on different servers that were running applying updates at the same time. I'd seen that several times, where we'd be deploying changes at the same time the apt mirrors were updating. 2. `Dockerfile`s weren't reproducible because the entire world around you was changing. Files would disappear or change from URLs. Base images would change randomly. It was common practice to run `apt-get update` in your image, and who knows what that was doing? At conferences, I started talking about how Docker images were not enough because of how much the world was changing under our feet. How Chef and Puppet and similar tools weren't enough because of how much of the system was unmanaged. How the root of this problem is how blind we were as operators were to the details of our software and systems. I remember paraphrasing Carl Sagan a lot: > If you want to reproducibly build a server from scratch, you first have to pin the universe. ### I've seen these words before... In January 2016 I was complaining to a close friend about how slow it was to build a new AMI with Chef. She pointed me to the [NixOS website][old-nixos-org]. I read it—and I was Not Impressed:tm:. Beyond the boring website, it said words I'd seen before. Those words I'd read about Chef and Docker. "Declarative." "Reliable." "Roll back." Yeah, yeah. Those words were heavily in vogue at the time. I'd heard them before, and they were lies. But my friend persisted. "I don't know, Graham... You should probably try it..." ### I tried it By the time I found [NixOS], I knew what a system needed to look like if it were to be reproducible, declarative, effective, and to actually accomplish these goals we were trying to achieve as an industry: 1. Strictly enforcing that all inputs are hashed. 1. Eliminating network access during the build. 1. Enabling you to build the system you want instead of trying to change a running system. And here it was. Nix made sense. Nix was fast. Nix was declarative. Nix was reliable. Nix's rollbacks really did work. I "got it." I was hooked. This was the future. I had to *make this* the future. With all love and respect to the Chef and Docker ecosystems, which taught me so much and to this day extend great generosity: I never wanted to use either tool again. ## All in I tried NixOS on a Wednesday and by Saturday I had erased macOS and switched to NixOS. {/* vale off */} It wasn't easy! {/* vale on */} Within a few weeks, I'd replaced thousands of lines of Chef cookbooks with a few hundred lines of Nix expressions and a Bash script to build and upload AMIs to AWS. The next step for me was fixing the obvious problem: Nix was hard to adopt. ### Packaging was hard A major problem was packaging software in the first place. So much of the software ecosystem around Nix hadn't even started [pinning] dependencies or writing out version locks. In a sense, the world was starting to realize what Eelco had identified over ten years prior: that pinning versions and hashing dependencies was not just worth doing, but critical to software security. This has been a big focus of the industry's work in the last fifteen plus years, but at the time there was so much to do. We're in a different place than we were at the time. {/* vale off */} The industry has moved "towards" Nix in so many ways, cutting the packaging difficulty from "impossible" to "easy" for so many languages. {/* vale on */} ### Documentation was hard The docs were written largely by academics or folks so deeply steeped in Nix their writing was impenetrable to outsiders. This has improved since but there is still plenty to be done. This problem was not just practical "how-to" documentation. Nix didn't have great documentation to communicate and teach why Nix was important and why its rules were important. It struggled to educate the world about the new way of working that it demanded and about the incredible gifts it had bestowed on its users. I would not have fallen in love with it if I had not already made the conceptual leaps I'd already made. ### Persisting was so rewarding Everything about Nix was a hugely rewarding puzzle. Every successful build was a huge dopamine hit, rewarding the challenges of navigating the Nix libraries and fixing build problems. Every pull request to [Nixpkgs] was quickly reviewed, merged, and *celebrated* by a relatively small group of dedicated folks who loved this tech as much as I did. ## Even more "in" After a couple years of intensely contributing to the project in my spare time, I quit my job to do full-time Nix consulting. Consulting taught me some important things. ### Consulting revealed that Nix's flexibility made it too hard As a Nix consultant, I would often drop into a project and team that had more or less successfully and effectively adopted Nix. Invariably, that team used Nix in a way that was wholly their own and completely unlike that of any other team I'd worked with. Their Nix champion had followed a choose-your-own-adventure game of writing scripts and tools and `.nix` files to adopt Nix at their company. But this was not their fault. There were no standard processes, no workflows, no off-the-shelf techniques to adopt and be successful. On top of that, a lot of Nixpkgs' interfaces were undocumented and hard to discover. ### Consulting revealed that Nix just wasn't quite mature enough Nix's flexibility meant joining a new team meant learning their own unique approach. As I worked with more and more serious use cases, I noticed a schism in expectations and reality. These problems compounded as I moved up into increasingly sensitive and critical use cases. They loved what Nix got them but they would squirm when we talked about how pull requests were merged. How casual the review process was. How casual the "maintainer" role was treated. How casually security patching was handled. It was uncomfortable to beat around the bush and say "well, historically it has been fine ..." and "folks are usually careful about updates..." And for security patching, corporate users had security and compliance objectives that didn't run on vibes alone. They needed to know what was vulnerable and what wasn't. They needed to know when security updates were available and how to get them. ## Let's get mature I became obsessed with maturing the project. I wrote [OfBorg] to catch more contributor errors before they merged. I started the NixOS security team and ran weekly security round-ups until our data source literally shut down. I wrote security advisories. I rewrote the Nix installer to be more defensive and reliable. I worked like crazy to make the Nix installer use the daemon for better security, and to be more defensive and reliable. The bones of that installer are still the foundation of today's upstream Nix installer. To be clear, while I did start and lead these (and other) initiatives and my fingerprints of effort and love are all over the project, I didn't do any of this alone. Thank you, to the entire community, for which all of this would be impossible. ## The world *needs* Nix I remember the moment I realized I was white-knuckling the project into maturing. Something in Nixpkgs broke. It wasn't a particularly big deal but it did need to be treated seriously and fixed. Someone made a comment like "good thing this isn't being used by banks or whatever!" At that moment, I was working with financial firms using Nix in their banking and trading platforms. I was dizzied by the disparity between how important the project was compared to the concrete reality of the project. The next 100 years of computing needs to start with Nix. Our world's infrastructure should not be stuck in a "don't touch it, it works" mode, but it is. And I don't just mean low-stakes systems. Our entire world is software, and as an industry we're failing the world. Critical infrastructure running power grids and nuclear reactors and robots and scientific research is woefully insecure because it is {/* vale off */}simply{/* vale on */} too hard and scary to maintain. Nix can, and does, solve many of these problems in real terms today. I felt at the time that it was a huge tragedy that Nix wasn't mature enough to be used in those scenarios. I felt frustrated and occasionally angry that so many people saw the sparkling diamond in front of us but were afraid of reaching for the big picture. This stuff matters. ### I love Nix. I hate that Nix doesn't have consistent interfaces. Nix was so obvious and important. And painful. The inconsistencies really stacked up. Not just across teams and projects and repositories, but across environments. Every new target was a fresh batch of misery. Setting up CI, or deploying to a new machine or cloud provider or bare metal server was a fresh batch of tiresome questions: * How do I onboard my team members? * How do I set up my cache in CI? Or on the new server? * How do I deploy to this machine? * How do I scale up and down quickly? * How do I get private repositories in there? * Why do I have to SSH in as root to deploy? * How do I manage my NixOS infrastructure that feels like 2015 and not 2005? It just went on and on and the answers invariably involved writing way too many Bash scripts predicated on hopes and dreams. Maintenance was annoying and the trade-offs were deeply sub-optimal. It is mind numbing how badly we were missing the table stakes of similar ecosystems. I wanted a developer experience that didn't involve banging together hundreds of lines of copy-pasted Bash scripts to be good. Nix's raison d'être is to make it possible to move software from one computer to another and have it keep working. The promise was there, it worked, but the reality was occasionally misery. #### Okay, fine, flakes I didn't care for adopting [flakes], but only because they were experimental. My thinking evolved. I came around after realizing that flakes cut down on inconsistencies by making a standard evaluation model and input/output interface for projects. Flakes are a first, small step, but a [foundationally important step][flakes-statement]. But a portion of the community has not made that leap yet. I still think that even if flakes are occasionally annoying, or even if there are flaws in their design, they're predictable, improve build performance, add stability, and facilitate collaboration across developers and teams. They make Nix viable, and delightful, at work. There are blog posts out there that describe how to get flakes without flakes. They're wrong because the interface is the point. Everything else is details. Today, we still find ourselves sitting on the most powerful technology of our lifetimes and we can't even decide on small steps in the direction of making it {/* vale off */}easier{/* vale on */} for folks to adopt it. ## Determinate, to live up to the promise of our industry So here we are. We're building it. We're unabashedly making Nix the best way to build and deploy software to the most important targets in the world. Our work means [flakes are enabled][nix-installer]. That a new user has a one-step process to get [private repositories][private-flakes] and access to their team's [cache]. We've made tools for tracking and updating dependencies. Our documentation, tools, and workflow are designed top to bottom to make adopting Nix dramatically more straightforward. We're working with IT and security teams to make Nix a welcome addition. We're deliberately cutting out puzzles and bash scripts for deployment and access control in dev, test and prod. We're building that smooth path from "Nix could solve these problems" to "Nix is solving these problems". We're making it possible to use Nix for critical software, with components that fit together in straightforward ways. We're helping teams focus on the valuable problems they're solving. We're enabling teams to safely take big leaps with confidence and safety. That's why we're here. That's our vision. To make amends and live up to the promise our industry has made to the world. [cache]: /blog/flakehub-cache-and-private-flakes [chef]: https://chef.io [determinate]: https://docs.determinate.systems [docker]: https://docker.com [flakes]: https://zero-to-nix.com/concepts/flakes [flakes-statement]: /blog/experimental-does-not-mean-unstable [nix-installer]: https://github.com/DeterminateSystems/nix-installer [nixos]: https://zero-to-nix.com/concepts/nixos [nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs [ofborg]: https://github.com/NixOS/ofborg [old-nixos-org]: https://web.archive.org/web/20160115032501/https://nixos.org [packer-ami]: https://grahamc.com/blog/packer-ami-device-volume-types [pinning]: https://zero-to-nix.com/concepts/pinning [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [puppet]: https://puppet.com [z2n-nix]: https://zero-to-nix.com/concepts/nix --- **Nix at work: FlakeHub Cache and private flakes** Published: October 23, 2024 URL: https://determinate.systems/blog/flakehub-cache-and-private-flakes Last year, we here at Determinate Systems [announced FlakeHub][flakehub-announcement], a platform for publishing and discovering [Nix flakes][flakes]. From the outset, FlakeHub provided pathbreaking features like [semantic versioning][semver] (SemVer) for flakes and a variety of ways to [discover][discovery] flakes published to the platform. In the meantime, numerous [Nix] users have published [hundreds of flakes][all-flakes] and used those flakes countless times—for their [Home Manager][hm] and [nix-darwin] configurations, to deploy [NixOS] systems to production, to distribute tools like [Tailscale] and [Colmena], and much more. Today, we're delighted to announce a new chapter for FlakeHub with the general availability of two new features: [**FlakeHub Cache**](#flakehub-cache) and [**private flakes**](#private-flakes). These powerful additions are now open to *all* FlakeHub organizations, helping you solve a variety of complex challenges that can arise when scaling [Nix] adoption across teams and projects. ## FlakeHub Cache We [announced the private beta][beta] of FlakeHub Cache a while back and as of today anyone can [sign up][signup] and [get started][cache-docs]. Traditional [Nix binary caches][caching] present significant challenges for large teams, often requiring them to manage multiple caches across projects, generate and distribute static credentials, and implement complex access controls on their own. As organizations grow, a number of problems compound: * Specific team members need access to specific subsets of [flake outputs][outputs] and artifacts * CI systems require secure, automated authentication * Security teams need audit trails and compliance controls * Using multiple caches can fragment build processes and slow down development FlakeHub transforms this experience by providing a cache designed for teams. Instead of managing multiple caches with static tokens, FlakeHub Cache provides a secure, unified, identity-aware cache with fine-grained access controls that integrates with your [existing identity management systems](#authentication). ## Private flakes We're also excited to announce that [private flakes][private-flakes] are now generally available. Private flakes enable teams to securely share and reuse Nix expressions without exposing sensitive code or configurations. As with FlakeHub Cache, private flakes integrate seamlessly with your organization's existing [authentication flows](#authentication). Like all flakes on FlakeHub, you can only [publish private flakes][publishing] from trusted CI systems (which means no ad-hoc publishing). That currently includes [GitHub Actions][gh-actions] and [GitLab CI][gitlab-ci], but we plan to support other providers, like [CircleCI] and [Semaphore CI][semaphore], in the future. To publish a private flake on [GitHub Actions][gh-actions], for example, you can use the [flakehub-push] Action, set the [visibility] to `private`, and you're good to go: ```yaml {3} title=".github/workflows/publish.yml" - uses: DeterminateSystems/flakehub-push@main with: visibility: private ``` You can also use our [publishing wizard][wizard] to set up your Actions workflow. ## Federated authentication that makes sense \{#authentication} FlakeHub Cache and private flakes are, naturally, features that require authentication. But rather than relying on brittle auth patterns like long-lived static credentials, FlakeHub uses [JSON Web Tokens][jwt] (JWTs) to dynamically integrate with your existing identity infrastructure: * [GitHub Actions][gh-actions] and [GitHub Enterprise OIDC][gh-enterprise] * [GitLab CI/CD][gitlab-ci] for both cloud and self-hosted instances * [Microsoft Entra ID][entra] SSO (formerly Azure Active Directory) * [Amazon IAM][iam] role-based access * Short-lived JSON Web Tokens (JWTs) for automation When a machine authenticates using one of these methods, FlakeHub automatically knows to which organization the machine belongs—and to which flakes and cache slices it has access. This approach aligns with zero-trust security principles, significantly reducing the attack surface and lowering the risk of issues like accidental cache poisoning. ## Built for teams We know that different teams need different levels of access, so we've built out a robust policy engine undergirding FlakeHub Cache and private flakes. We have built internal features around this policy engine, like IP restrictions and granting individual devices [deploy-only access to specific flake outputs][deploy-only-outputs]. In the future, users will be able to write their own custom policies that let you: * Grant precise access to [flakes] and [flake outputs][outputs] based on team roles * Control artifact visibility between projects * Implement geographic access restrictions * Enforce compliance requirements * Monitor and audit [cache](#flakehub-cache) usage Unlike systems with simplistic, all-or-nothing access controls, FlakeHub's policy engine provides fine-grained cache access control, enabling organizations to grant scoped access to users, devices, and automated processes. This level of control is desirable—if not make-or-break—in highly regulated environments, supporting cross-team isolation and even geo-embargoes. ## Built to integrate \{#integrate} While FlakeHub works with any Nix installation, it truly shines when paired with [Determinate Nix][det-nix] as [`determinate-nixd`][dnixd] is designed with FlakeHub Cache and private flakes in mind, creating an experience that just works out of the box. Team members can authenticate to FlakeHub using the [`determinate-nixd login`][login] command, and CI pipelines and machines can automatically authenticate using their existing identity via `determinate-nixd login {aws|github-action|gitlab-pipeline}` (depending on the platform). The thoughtful integration between Determinate Nix and FlakeHub delivers an experience where builds are faster, security is stronger, and developers can focus on their work instead of managing Nix infrastructure. It's a practical demonstration of our vision for bringing Nix to professional teams, where every component is designed to work together seamlessly. ## Lightning-fast deployments FlakeHub also enables you to rapidly deploy fully evaluated [closures]. You can quickly deploy [NixOS] configurations, [nix-darwin] configurations, and [Home Manager][hm] environments, without needing Nix to be installed on the target system, using [fh], the CLI for FlakeHub. This command would apply a [NixOS] configuration to the current machine: ```shell title="Using fh apply to deploy a NixOS system" fh apply nixos AmbiguousTechnologies/ethercalc/0.1#nixosConfigurations.ethercalc ``` And this command would apply a [nix-darwin] configuration to the current machine: ```shell title="Using fh apply to deploy a nix-darwin system" fh apply nix-darwin AmbiguousTechnologies/ethercalc/0.1#darwinConfigurations.developer-workstation ``` Finally, this command would apply a [Home Manager][hm] configuration: ```shell title="Using fh apply to deploy a Home Manager configuration" fh apply home-manager AmbiguousTechnologies/profile/0.1#homeConfigurations.developer-environment ``` In both cases, fh fetches the closure associated with the resolved store path *directly* from FlakeHub Cache; it does *not* [realise][realisation] any derivation or even evaluate the referenced expression. That's because FlakeHub can **pre-generate store paths for you** and transfer computational responsibility from Nix to FlakeHub, which is particularly useful for deploying to resource-constrained devices. We'll say much more about this—and provide a powerful demo—in an upcoming post. And even though the `AmbiguousTechnologies/ethercalc` flake is [private](#private-flakes), this all works seamlessly as long as the machine is logged in using [`determinate-nixd login`](#integrate). The current machine has access to the flake and to the respective cached outputs; non-logged-in machines have access to neither. ## Complete package independence We were recently asked how FlakeHub Cache handles dependencies from [Nixpkgs] and other sources. FlakeHub maintains complete copies of all packages, including those available from [cache.nixos.org][cno]. This design choice serves two key purposes: 1. It respects the Nix community's shared resources. While we could save costs by deferring to [cache.nixos.org][cno], the upstream cache is a commons, and it isn't appropriate or respectful of those commons to save a few pennies in storage and bandwidth at the expense of the community project. 2. It ensures total reliability for your builds. The upstream cache can be garbage collected or potentially tampered with, which could break or compromise your builds. With FlakeHub Cache, your artifacts remain available for as long as you need them, independent of upstream availability. ## Available today FlakeHub Cache and private flakes are [available now][pricing] for all FlakeHub organizations. Pricing is straightforward: **$20** per FlakeHub organization member per month. For a limited introductory period, all storage and bandwidth costs are included at no additional charge to help teams get started, and then priced at cost after that. ## The road ahead FlakeHub Cache and private flakes are central to our mission to broaden Nix adoption, and this is just the start. We have more to come in the not-too-distant future, so [stay tuned][blog]. We are eager to learn how teams utilize these features to enhance their development workflows while ensuring compliance and adopting a strong security posture. Sign up now at https://flakehub.com and visit our [getting started docs][get-started] to begin taking advantage of these features in your workflows. And as always, we'd love to hear your feedback on our Discord at https://determinate.systems/discord. [all-flakes]: https://flakehub.com/flakes [beta]: /blog/flakehub-cache-beta [blog]: /blog/categories/announcement [cache-docs]: https://docs.determinate.systems/flakehub/cache [caching]: https://zero-to-nix.com/concepts/caching [circleci]: https://circleci.com [closures]: https://zero-to-nix.com/concepts/closures [cno]: https://cache.nixos.org [colmena]: https://flakehub.com/flake/zhaofengli/colmena [deploy-only-outputs]: https://docs.determinate.systems/guides/resolve-only-deploy [det-nix]: /blog/announcing-determinate-nix [discovery]: https://docs.determinate.systems/flakehub/discovery [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [entra]: https://www.microsoft.com/security/business/identity-access/microsoft-entra-id [fh]: https://github.com/DeterminateSystems/fh [flakehub-announcement]: /blog/introducing-flakehub [flakehub-push]: https://github.com/DeterminateSystems/flakehub-push [flakes]: https://zero-to-nix.com/concepts/flakes [get-started]: https://docs.determinate.systems/getting-started [gh-actions]: https://github.com/features/actions [gh-enterprise]: https://docs.github.com/en/enterprise-cloud@latest/admin/managing-iam/configuring-authentication-for-enterprise-managed-users/configuring-oidc-for-enterprise-managed-users [gitlab-ci]: https://docs.gitlab.com/ee/ci [hm]: https://github.com/nix-community/home-manager [iam]: https://aws.amazon.com/iam [jwt]: https://jwt.io [login]: https://docs.determinate.systems/determinate-nix/#determinate-nixd-login [nix]: /nix [nix-darwin]: https://github.com/LnL7/nix-darwin [nixos]: https://zero-to-nix.com/concepts/nixos [nixpkgs]: https://github.com/NixOS/nixpkgs [outputs]: https://zero-to-nix.com/concepts/flakes#outputs [pricing]: https://flakehub.com/pricing [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [publishing]: https://docs.determinate.systems/flakehub/publishing [realisation]: https://zero-to-nix.com/concepts/realisation [semaphore]: https://semaphore.io [signup]: https://flakehub.com/login [semver]: https://docs.determinate.systems/flakehub/discovery [tailscale]: https://flakehub.com/flake/tailscale/tailscale [visibility]: https://docs.determinate.systems/flakehub/concepts/visibility#private [wizard]: https://flakehub.com/new --- **Announcing Determinate Nix** Published: October 21, 2024 URL: https://determinate.systems/blog/announcing-determinate-nix Today, I'm excited to announce [**Determinate Nix**](/nix), [Determinate Systems][detsys]' distribution of [Nix] built for teams and optimized for the enterprise. Nix is extremely versatile and powerful, but as is often the case with free software projects, it is also unopinionated and "low policy." As a result, getting Nix to work well in a development team requires a frustrating amount of configuration. For instance: * Setting up access to a private binary cache requires figuring out access control, distributing credentials, and configuring files like [`nix.conf`][nix-conf]. * While [Determinate Nix Installer][installer] has greatly improved the installation experience on [macOS], enterprise users still lack features like Mobile Device Management (MDM) support, proper [Amazon EC2][ec2] integration and [Keychain] support. * Nix doesn't enable garbage collection by default, so users' disks have a tendency to fill up. With Determinate Nix, our goal is to transform Nix from what it is today—a tool with great potential but with too many hard edges to be ready for prime time—into a part of your stack that does the Right Thing *out of the box* for teams big and small. To begin making that a reality, the initial launch includes: * Automatic **binary cache configuration**: to get access to your organization's [FlakeHub binary cache][cache], you just need to run `determinate-nixd login` (more on Determinate Nixd below). * A much-improved experience on [**macOS**][macos]. For instance, Determinate Nix automatically uses certificates from the macOS [Keychain], and it supports [fully automated installation on AWS EC2 instances][macos-ec2]. * Periodic **garbage collection** quietly guards your system against Nix store bloat. * **Amazon Web Services integration**: Determinate Nix can automatically log in to FlakeHub using [AWS IAM][iam] roles. Determinate Nix is *not* a fork of Nix—it is a downstream Nix distribution. Its features are implemented through a separate daemon called [Determinate Nixd][dnixd]. It's written in [Rust] (for the sake of memory safety) and it supervises the regular [Nix daemon][nix-daemon] while also providing some [other utilities][dnixd]. Our Nix distribution is carefully vetted to ensure compatibility and stability, guided by the telemetry collected by our [Determinate Nix Installer GitHub Action][installer-action]. ## The big picture Determinate Nix is part of a broader product experience that we call **Determinate**, which you'll be hearing much more about in the coming days. Our goal for Determinate is to enable fearless innovation by bringing Nix to teams, providing a complete Nix-based workflow from installation through collaboration and CI to deployment. The other central component of Determinate is [**FlakeHub**][flakehub], a service that provides a place for teams to [privately][private-flakes] publish flakes. It provides a binary cache called [**FlakeHub Cache**][cache] that supports fine-grained access control policies as well as support for [private flakes][private-flakes]. But to use FlakeHub and FlakeHub Cache on developer workstations and in CI requires a fair amount of error-prone configuration when you're using regular Nix. So one of the main reasons why we created Determinate Nix is to make Nix "just work" with the Determinate platform. We'll talk more about private flakes, binary cache, and the Determinate big picture in upcoming blog posts! ## Future plans We will continue to add new features to Determinate Nix to make the Nix user experience ever smoother for teams. These include better authentication support for flakes and binary caches, [flake schemas][schemas], [parallel evaluation][parallel], and much more. ## Getting Determinate Nix If you're using the Determinate Nix Installer, then getting Determinate Nix is as straightforward as adding the `--determinate` flag to the installation command: ```shell title="One-liner for installing Determinate Nix" curl --proto '=https' --tlsv1.2 -sSf -L \ https://install.determinate.systems/nix | \ sh -s -- install --determinate ``` For NixOS users, we provide a [flake][determinate-flake] that makes switching to Determinate Nix straightforward. For more information on installation and use, see the [Determinate documentation][getting-started]. We're interested in your feedback and would love to hear from you on our Discord at https://determinate.systems/discord. [cache]: https://docs.determinate.systems/flakehub/cache [determinate-flake]: https://flakehub.com/flake/DeterminateSystems/determinate [detsys]: https://determinate.systems [dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd [ec2]: https://aws.amazon.com/ec2 [flakehub]: https://flakehub.com [getting-started]: https://docs.determinate.systems/getting-started [iam]: https://aws.amazon.com/iam [installer]: https://github.com/DeterminateSystems/nix-installer [installer-action]: https://github.com/DeterminateSystems/determinate-nix-action [keychain]: https://developer.apple.com/documentation/security/keychain-services [macos]: /blog/tags/macos [macos-ec2]: /blog/unattended-nix-install-macos-aws-ec2 [nix]: https://zero-to-nix.com/concepts/nix [nix-conf]: https://nix.dev/manual/nix/latest/command-ref/conf-file [nix-daemon]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-daemon [parallel]: /blog/parallel-nix-eval [private-flakes]: https://docs.determinate.systems/flakehub/private-flakes [rust]: https://rust-lang.org [schemas]: /blog/flake-schemas --- **Discontinuing support for macOS Monterey** Published: October 9, 2024 URL: https://determinate.systems/blog/nix-installer-macos-12-monterey Determinate Nix Installer and related software will no longer support macOS Monterey (version 12). This change follows Apples' standard procedure of discontinuing support for older macOS versions. ### Timeline Our software will no longer explicitly target or test against macOS Monterey starting **Monday, October 21, 2024**. Our deprecation date is in advance of [GitHub's own deprecation timeline][github], which drops support on December 3, 2024. Note that between October 21 and December 3, [`DeterminateSystems/nix-installer-action`][nix-installer-action] and [`DeterminateSystems/magic-nix-cache-action`][magic-nix-cache-action] will detect macOS Monterey, and fall back to the last release known to support macOS Monterey. Security patches will not be back ported to these older releases. This fallback behavior will be removed after GitHub fully disables hosted runners for macOS Monterey. ### Affected Software * nix-installer * fh * flake-checker * flakehub-push * determinate-nixd * magic-nix-cache ### How will this affect users? According to our data, most macOS Monterey users are on GitHub Actions. These users will need to update their workflows to use a more recent macOS version, like `macos-latest`. But we understand that this change might impact macOS Monterey users we're unaware of, as future updates to our software will no longer explicitly support this platform. We're here to help. If you do need continued support for macOS Monterey, please reach out to us at hello@determinate.systems. Our goal is to ensure that your projects continue to run smoothly and we're open to finding ways to accommodate your requirements. We would be happy to discuss your needs and explore potential solutions. [github]: https://github.com/actions/runner-images/issues/10721 [nix-installer-action]: https://github.com/DeterminateSystems/nix-installer-action [magic-nix-cache-action]: https://github.com/DeterminateSystems/magic-nix-cache-action --- **Fully automated Nix installation for macOS on AWS EC2** Published: October 1, 2024 URL: https://determinate.systems/blog/unattended-nix-install-macos-aws-ec2 Installing Nix on macOS has been [largely solved][announcement-determinate-nix-installer] for some time now, but a quirk of [Amazon Web Services][aws]' macOS support [created a huge wrinkle][upstream-bug] for those running macOS on AWS. This quirk meant that users needed to graphically log in over VNC to manually approve "full disk access" to the Nix daemon. This wrinkle is now smoothed out, and the installation is fully automatable. Users of the [Determinate Nix Installer][installer] can now install Nix to macOS on Amazon Web Services without needing to interact with any graphical user interface. ## Unlocking new use cases like autoscaling macOS Previously, Nix users couldn't autoscale macOS on AWS—after all, there's nothing "auto" about manually approving full disk access! With automated Nix installation, autoscaling is seamless, allowing for larger and more flexible AWS deployments. ### Appropriate use cases and limitations The new fully unattended installation isn't appropriate for all use cases, as this new behavior brings some limitations to your instance lifecycle. | Use case | Status | |-------------------------------------------------------------------------------|-------------------| | Ephemeral macOS instances that terminate when the machine is no longer needed | Fully automatable | | Auto-scaled macOS infrastructure | Fully automatable | | Long-term macOS instances that are stopped and re-started | Do not automate | | macOS instances that are snapshotted and cloned | Do not automate | See the caveats [below](#caveats). ## How to install Nix on macOS in EC2 Run this install command: ```shell title="Install Nix" {4,5} curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix \ | sh -s -- install macos \ --no-confirm \ --determinate \ --use-ec2-instance-store ``` ## Why is macOS on AWS different? Amazon's Apple hardware boots macOS from an EBS volume which is presented over PCIe. Because this volume isn't the soldered-in hardware, macOS considers it to be a removable volume. macOS' permissions model requires software accessing removable volumes to be granted special privileges. The Determinate Nix Installer creates a new volume for the Nix Store on the same disk as the operating system. That new volume is considered "removable" even though it is on the same disk. ## How does it work? The core change we've made is that we've added a `--use-ec2-instance-store` flag for installing Determinate Nix. When this flag is set, the installer installs Nix to a volume on the internal disk, eliminating the need for manual approval. Note that this feature is limited to Determinate users (`--determinate`) due to runtime orchestration provided by `determinate-nixd`. ### Caveats Setting the `--use-ec2-instance-store` flag installs Nix to the instance's ephemeral instance store. Using the instance store means that: * The Nix Store is erased when the machine is stopped. * The Nix Store is not captured by EBS snapshots. * Standard macOS reboots are perfectly safe. Please see [Data persistence for Amazon EC2 instance store volumes][data-persistence] for further details. ## Ready to automate your macOS CI pipeline? We've built a comprehensive guide that takes this announcement to the next level, with step-by-step instructions for setting up auto-scaling macOS CI infrastructure with AWS IAM authentication. Learn how to: * Deploy with EC2 launch templates * Configure secure IAM authentication with FlakeHub * Set up auto-scaling groups for elastic CI capacity * Integrate with AWS Systems Manager for management **[Check out our detailed deployment guide →](https://docs.determinate.systems/guides/deploy-nix-macos-ec2)** Join the teams already using Determinate Nix on AWS to build faster, more secure CI pipelines. Have questions? Join us on [Discord](https://determinate.systems/discord) where our team and community can help you get started. [announcement-determinate-nix-installer]: /blog/determinate-nix-installer/ [aws]: https://aws.amazon.com [data-persistence]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-store-lifetime.html [installer]: https://github.com/DeterminateSystems/nix-installer [upstream-bug]: https://github.com/NixOS/nix/issues/6291 --- **Solving corporate TLS certificates for Nix on macOS** Published: September 27, 2024 URL: https://determinate.systems/blog/zscaler-macos-and-nix-on-corporate-networks Enterprises often use tools like [Zscaler] to bolster their overall security posture. Zscaler is an intercepting TLS proxy that inspects the traffic coming in and going out of the network. Because it intercepts *all* traffic, clients have to be configured with a custom TLS certificate for the connections to be trusted. This doesn't always play nicely with other tools, especially tools that require TLS certificates—like [Nix]. ## Zscaler and Nix used to be a big pain Nix users in enterprise environments have run head-first into this problem countless times: ```log warning: unable to download '...': SSL peer certificate or SSH remote key was not OK (60) ``` On Linux, this isn't a huge issue. Users and admins can add the certificate to their trusted bundle at a standard location like `/etc/ssl/certs/ca-certificates.crt` or `/etc/pki/tls/certs/ca-bundle.crt`. On macOS, however, the story is slightly more annoying for Nix because macOS stores the custom certificates in [Keychain], a key and secret store used by applications across the macOS ecosystem. If you're fully steeped in macOS, this is fine and works out of the box. But if you're using a tool like Nix that's mostly designed and used by Linux users to build Linux software then... things can get unpleasant. Users have long had to export their enterprise certificate from Keychain, reconfigure Nix, and manage this over time. When the certificate eventually expires and needs rotating, the user has to figure out how they fixed it the first time and try again. {/* vale off */} *Uff da*... what a headache. {/* vale on */} ## Determinate means Nix + Zscaler doesn't have to be a huge pain Our goal with [Determinate] is to make Nix a pleasure to use across these platforms and to elegantly solve thorny problems for both users and IT administrators. So we fixed the Nix/Zscaler certificate issue on macOS. Determinate automatically configures Nix on macOS with an up-to-date certificate bundle from Keychain and synchronizes the bundle with Keychain over time. ## How to install Determinate Install Determinate with the [Determinate Nix Installer][installer] by passing the `--determinate` flag: ```shell curl --proto '=https' --tlsv1.2 -sSf -L \ https://install.determinate.systems/nix | sh -s -- install --determinate ``` Alternatively, you can use our [signed macOS `.pkg`][macos-pkg] for convenient distribution. If [Installomator] is your thing, we provide a [Mobile Device Management (MDM) script][mdm-script] that integrates especially well with automated processes. Determinate is available on all systems that the Determinate Nix Installer supports. ## More to come This is just the beginning of a series of posts about what we're doing to improve the experience of using Nix in the enterprise. If this is music to your ears, join our Discord at https://determinate.systems/discord and come chat. [determinate]: /nix [Installomator]: https://github.com/Installomator/Installomator [installer]: https://github.com/DeterminateSystems/nix-installer [keychain]: https://support.apple.com/guide/keychain-access/welcome/mac [macos-pkg]: https://docs.determinate.systems/getting-started/#determinate-pkg [mdm-script]: https://docs.determinate.systems/advanced/deploy-with-mdm [nix]: https://zero-to-nix.com/concepts/nix [zscaler]: https://zscaler.com --- **Prepare Nix for macOS Sequoia** Published: September 13, 2024 URL: https://determinate.systems/blog/nix-support-for-macos-sequoia Determinate Nix Installer v0.26.0 has been released, including a `repair sequoia` command that addresses issues with the upgrade path to macOS Sequoia. Before upgrading to Sequoia, Nix users should run the repair tool: ``` curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix/tag/v0.26.0 | sh -s -- repair sequoia --move-existing-users ``` Running the repair tool after upgrading to Sequoia will work, too, but Nix will not work until after the repair tool is run. ### Who needs to run the repair tool? Nix users on macOS who installed Nix with the upstream Nix installer, or Determinate Nix Installer [`v0.22.0`](https://github.com/DeterminateSystems/nix-installer/releases/tag/v0.22.0) and older. Run `/nix/nix-installer --version` to identify the Determinate Nix Installer version used on your system. The repair tool is safe to run, even if you installed with a Determinate Nix Installer version newer than [`v0.22.0`](https://github.com/DeterminateSystems/nix-installer/releases/tag/v0.22.0). ### Can I repair my Nix installation that wasn't installed by the Determinate Nix Installer? Yes. The Determinate Nix Installer's repair tool works on any Nix installation on macOS. #### Caveat for advanced users Users who customized their Nix build users should read the help output of the `repair sequoia` command by passing the `--help` flag. ### Why is this repair tool required? macOS Sequoia creates users with `UniqueID`s that conflict with Nix's `_nixbldN` (that is `_nixbld1`) users. When upgrading to Sequoia, macOS' update tool deletes the conflicting users. ### More information For more information, see the upstream Nix tracking issue: https://github.com/NixOS/nix/issues/10892 Or, join our [Discord](https://determinate.systems/discord) and chat with the team. --- **Discontinuing support for i686-linux in the Determinate Nix Installer** Published: July 24, 2024 URL: https://determinate.systems/blog/nix-installer-i686-linux Here at [Determinate Systems][detsys], our mission is to transform the software industry by bringing [Nix] to the enterprise. As a small team, we periodically evaluate how we spend our time and the impact that we have on our users. Today, we're announcing that we plan to discontinue supporting `i686-linux` in the [Determinate Nix Installer][installer]. The decision to end support for `i686-linux` comes after carefully reviewing usage patterns and the resources required to maintain this system type. For `i686-linux`, we saw only *one* attempt to install Nix in the last three months—and that attempt failed. Given this essentially non-existent usage, the burden of continuing to support and test compatibility significantly outweighs the benefits. ### Timeline Determinate Nix Installer releases will continue to support `i686-linux` until **August 15, 2024**. ### How will this affect users? According to our data, **it won't**. But we understand that this change might impact `i686-linux` users we're unaware of, as future updates to the Determinate Nix Installer will no longer support this architecture. We're here to help. If you do need continued support for `i686-linux`, please reach out to us at hello@determinate.systems. Our goal is to ensure that your projects continue to run smoothly and we're open to finding ways to accommodate your requirements. We would be happy to discuss your needs and explore potential solutions. The [Determinate Nix Installer][installer] will continue to support these systems for the foreseeable future: * `x86_64-linux` * `aarch64-linux` * `x86_64-darwin` * `aarch64-darwin` [detsys]: / [installer]: https://github.com/DeterminateSystems/nix-installer [nix]: https://zero-to-nix.com/concepts/nix --- **Parallel Nix evaluation** Published: June 27, 2024 URL: https://determinate.systems/blog/parallel-nix-eval The Nix user experience is significantly affected by the speed of the [Nix] expression evaluator. [Nixpkgs] and [NixOS] have grown massively in recent years, and continue to grow. This means that operations like [`nix search nixpkgs`][search] or evaluating NixOS system configurations take increasing amounts of time. For instance, listing all packages (`nix-env -qa`) took less than 3 seconds in Nixpkgs 16.03, but more than 15 seconds in 24.05; evaluating a small NixOS configuration (`closures.lapp.x86_64-linux`) finished in less than half a second in 16.03 but needs more than 3 seconds today. Similarly, many users have large [flakes] that take a long time to evaluate, either in CI or on their own systems (for example when running [`nix flake check`][nix-flake-check]). Many improvements have been made to the Nix evaluator in recent times, but one obvious avenue for improvement has not been taken so far: *parallel evaluation*. The Nix evaluator has always been single threaded, which means that Nix hasn't been able to utilize all those wonderful CPU cores available on modern systems—yet. At [Determinate Systems](/) we've recently implemented a parallel evaluator, currently available in [a draft PR][draft]. It already gives a substantial performance improvement—speedups of a factor 3-4 are typical. In this blog post, we describe some of the work that was needed to achieve parallelism, preceded by a short overview of the inner workings of the evaluator. ## How Nix evaluation works In the Nix evaluator, every value is represented as a 24-byte (3-word) structure that consists of a *type* field and a type-dependent *payload*. The type is a tag such as "integer", "Boolean" or "lambda." For example, the attribute ```nix x = 2; ``` produces the value In principle, purely functional languages like Nix are ideal for parallel evaluation, because values are immutable. This means that the pointer to the value of `x` can be freely shared between threads, as we don't have to worry about the value getting updated by one thread while another is accessing it. There is one exception, however, to the immutability of values: *thunks*. These are the mechanism by which lazy evaluation is implemented. A thunk represents a delayed computation, or *closure*, that is *overwritten in place* by the result of the computation, if and when the result is needed. A thunk consists of two pieces of information: * A pointer to an abstract syntax representation of the definition of the value (in compiled languages, this would be a pointer to the compiled code, but the Nix evaluator is an *interpreter*, not a compiler). * A pointer to the *environment*, which is a data structure that records all the variables that are in scope. In an expression like `let x = 1; in let y = 2; in `, for instance, the environment of the thunk for expression `` maps the variables `x` and `y` to their respective values. As an example, consider the expression ```nix let x = 2; y = 3; z = x * y; in z ``` The value `z` is initially represented as a thunk: If `z` is never needed, then `z` will forever remain a thunk. But if another expression needs `z`, the thunk will be *forced*, meaning that the expression `x * y` will be computed using the environment that maps `x` to 2 and `y` to 3, and thunk is overwritten: Any subsequent use of `z` will see the final value `6`, so `z` will be evaluated at most once. ## Making the evaluator thread safe To speed up operations like `nix search nixpkgs`, we would like to distribute the work of evaluating the attributes in Nixpkgs across multiple threads. As noted above, only thunks represent a difficulty; "final" values like integers or attribute sets can be accessed in parallel since they're never updated. To handle thunks correctly and efficiently, we have these requirements: * No thread should ever see a "half-updated" thunk, for example one where the "type" field has been changed to "integer" but the integer payload is in an undefined state. * A thunk should not be evaluated more than once. Thus, if a thunk is being evaluated by one thread, other threads that need the result of the thunk should wait (or do other work). * Support for multi-threaded evaluation should not have a negative impact on single-threaded performance or memory consumption. This means, for instance, that we can't just wrap every value in a [mutex]. In addition, the changes should be minimally invasive to the current evaluator. The approach we took is fairly straightforward: * The "type" field in values is now an [`std::atomic`][atomic]. This allows its state transitions to be done using atomic compare-and-swap instructions, and enables the necessary compiler and CPU memory barriers that ensure that threads see changes to values in the right order. * The payload of a thunk value is updated to its final value *before* the type is updated. Thus, if a thread sees a value with an immutable type (for example "integer"), then its payload (for example the integer value) is also valid. * There are three new value types: * **Pending** denotes a thunk that is being evaluated by a thread, and has no other threads waiting for it. * **Awaited** denotes a thunk that is being evaluated by a thread, and has other threads waiting for it. * **Failed** denotes a thunk whose evaluation threw an exception. Its payload stores an [`std::exception_ptr`][ptr] that allows other threads to re-throw the exception if needed. * The lifecycle of a thunk is as follows: * The value is initialized as a thunk. * When a thread forces a value that has type "thunk", it sets its type to "pending". This is done using an atomic compare-and-swap to ensure that only one thread will actually evaluate the thunk. * When a thread forces a value that has type "pending", it registers itself in a list of waiters (stored separately from the value) and sets the type of the value to "awaited" (using an atomic compare-and-swap to handle the case where the first thread has just finished the value). It then waits to be woken up by the first thread when it has finished with the value. * When a thread finishes evaluating a thunk, it updates the payload of the value, and then does an atomic swap to change the type to its final type. If the previous type was "pending", that's all it needs to do, since no other threads are waiting. If it was "awaited", it wakes up the threads that are waiting. Beyond thunk handling, a couple of other data structures in the evaluator had to be made thread-safe. Usually it was sufficient to wrap them in a mutex, but some data structures are highly contended and so required special handling to make them performant. In particular the *symbol table*, which ensures that every identifier is stored in memory only once, was rewritten to make symbol access lock-free. ## Results Currently only two `nix` subcommands distribute work across multiple threads. This command shows the contents of the `nix` flake: ```bash nix flake show --no-eval-cache --all-systems --json \ github:NixOS/nix/afdd12be5e19c0001ff3297dea544301108d298 ``` On a Ryzen 5900X (12 cores), the single-threaded evaluator takes **23.70s**. The multi-threaded evaluator brings this down to **5.77s** (using 12 threads), a **4.1x** speedup. This command searches the contents of Nixpkgs: ```bash nix search --no-eval-cache github:NixOS/nixpkgs/bf8462aeba50cc753971480f613fbae0747cffc0 ^ ``` This went from **11.82s** to **3.88s** (using 16 threads), a **3.0x** speedup. This is less than for the previous benchmark because there are more shared dependencies (for example, most threads cannot make progress until the `stdenv` value has been computed). ## Trying it out The multi-threaded evaluator still has a couple of concurrency bugs that need to be sorted out, but if you do want to try it out, you can get it by running: ```bash nix shell github:DeterminateSystems/nix-src/multithreaded-eval ``` You then need to set the environment variable `NR_CORES` to the number of threads that Nix should use. Here's an example: ```bash NR_CORES=8 nix search --no-eval-cache nixpkgs hello ``` ## Next steps In the near future, we will make more Nix subcommands multi-threaded, such as [`nix flake check`][nix-flake-check]. In addition, evaluations of single-flake output attributes such as NixOS system configurations should exploit parallelism automatically. One way to do this would be to distribute derivation attributes across threads. [atomic]: https://en.cppreference.com/w/cpp/atomic/atomic [draft]: https://github.com/NixOS/nix/pull/10938 [flakes]: https://zero-to-nix.com/concepts/flakes [mutex]: https://en.wikipedia.org/wiki/Lock_(computer_science) [nix]: https://zero-to-nix.com/concepts/nix [nix-flake-check]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-flake-check [nixos]: https://zero-to-nix.com/concepts/nixos [nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs [ptr]: https://en.cppreference.com/w/cpp/error/exception_ptr [search]: https://zero-to-nix.com/start/nix-search --- **Nix as a WebAssembly build tool** Published: June 13, 2024 URL: https://determinate.systems/blog/nix-wasm I've been pretty bullish on [**WebAssembly**][wasm]—or **Wasm**—for quite some time, as I believe that it offers a degree of portability and operational simplicity that goes beyond that of [Linux virtual machines][vms] and even [OCI containers][containers]. Run it in the browser, run it on your laptop, run it on [Kubernetes][k8s], run it on a dedicated Wasm platform like [Fermyon]. While I'm not convinced that it will fully *supplant* VMs or containers any time soon, I do think that there's a strong case that Wasm is already a superior technology in domains like [edge functions][edge] and [platform extensions][extensions]. But alas, there's a bit of a snag: Wasm is extremely portable *once you've already built it*. But building Wasm isn't trivial for three reasons: * You can compile many languages to Wasm and each has its own tools and approaches. * There are numerous Wasm-specific tools that you may want to include in your toolchain, such as [wasm-tools] and the [WebAssembly Binary Toolkit][wabt] (WABT). * There are several Wasm runtimes currently available, including [WasmEdge] and [Wasmtime]. And any time you need a bunch of separate tools to get your work done, there's room for error and the classic "read the README and use apt-get/Homebrew/whatever to create your environment" approach to dependency management quickly runs into hard limits. Unsurprisingly, I think that using [Nix] to package Wasm provides a compelling path forward here. ## My example project To show how you can use Nix to work with Wasm, I've created an example project in the [DeterminateSystems/nix-wasm-example][example] repo. The actual [Wasm app][app] I build here is extremely basic: just a [Rust] program that outputs the string `"Hello there, Nix enthusiast!"` What's notable here is that the program is compiled to conform to the [WebAssembly System Interface][wasi] (WASI), which essentially means that it's built to interact with the outside world (by default, WebAssembly can't act as a command line interface or system tool). To build the program into WASI-compliant Wasm, I created a special Nix function called [`buildRustWasiWasm`][build-wasi] that wraps [Naersk]'s `buildPackage` derivation function. That function builds the Rust sources using a special Rust toolchain that includes the [`wasm32-wasip1`][target] target. The post-install phase of the derivation also uses [`wasm-strip`][wasm-strip] to make the final Wasm binary more lean and [`wasm-validate`][wasm-validate] to ensure that the resulting binary is valid. ```shell title="Build hello-wasm and inspect the result" nix build "https://flakehub.com/f/DeterminateSystems/nix-wasm-example/*.tar.gz#hello-wasm" ls ./result/lib ``` You should see a `hello-wasm.wasm` binary in that directory. Being able to build WebAssembly from Rust sources deterministically, without needing to use `apt-get` or Homebrew or anything else, is nice. But Nix enables you to do much more, so let's have a bit more fun. ## A full Wasm package The `nix build` command above deterministically builds a single Wasm binary from Rust source. Build this derivation and inspect the output: ```shell title="Build hello-wasm-pkg and inspect the result" nix build "https://flakehub.com/f/DeterminateSystems/nix-wasm-example/*.tar.gz#hello-wasm-pkg" tree result ``` That yields this filesystem tree: ```shell title="Filesystem tree for ./result" result ├── lib │ └── hello-wasm.wasm └── share ├── hello-wasm-dump.txt ├── hello-wasm.dist └── hello-wasm.wat 3 directories, 4 files ``` What you see here is a kind of WebAssembly package that includes not just the executable Wasm binary but also some information about it: * The `hello-wasm-dump.txt` file is produced by the [wasm-objdump] tool. It provides information about our binary, including headers, type definitions, function definitions, and more, which can be used by IDEs, debuggers, and other tools. * The `hello-wasm.dist` file is produced by the [wasm-stats] tool. It provides information about the size of sections, functions, and more, which can be used to optimize performance, to debug, and more. * The `hello-wasm.wat` file is a human-readable textual representation of the binary, built by the [wasm2wat]. Tools like [wat2wasm] can use these files to generate Wasm from that textual format and runtimes like [Wasmtime] can run them directly. This package is built using the [`buildRustWasmPackage`][build-wasm-pkg] function, which wraps the [`buildRustWasiWasm`][build-wasi] function mentioned above. There are plenty of other things we could add to such a "Wasm package," but this provides a small taste. ## A working CLI While you can run WebAssembly in the browser, in the cloud, on [Kubernetes][k8s], and in many other places, an emerging use case is running it as a CLI tool. What makes using Wasm as a CLI tool a bit tricky on a lot of systems is that you need to have a Wasm runtime present on the system to convert WASI-compatible Wasm into system calls. With Nix, we can directly solve this problem by creating derivations that use a Wasm runtime to run a compiled Wasm binary. Let's run our compiled binary using the [Wasmtime] runtime: ```shell title="Run the compiled binary using Wasmtime" nix run "https://flakehub.com/f/DeterminateSystems/nix-wasm-example/*.tar.gz#hello-wasmtime-exec" ``` Here, I created a [`buildRustWasmtimeExec`][build-wasmtime-exec] function that creates a wrapper script using [`makeWrapper`][wrapper] that runs [WasmEdge] and passes in a path to our compiled Wasm binary (all of this happen in the [Nix store][store], of course). You can also run the binary using the [WasmEdge] runtime: ```shell title="Run the compiled binary using WasmEdge" nix run "https://flakehub.com/f/DeterminateSystems/nix-wasm-example/*.tar.gz#hello-wasmedge-exec" ``` As with WasmTime, I created a [`buildRustWasmEdgeExec`][build-wasmedge-exec] function that creates a wrapper script that runs [WasmEdge] and passes in a path to our compiled Wasm binary. Congrats! You just ran two compiled Wasm binaries on your machine using two separate Wasm runtimes, with everything built deterministically and reproducibly using Nix. ## Beyond containers and VMs This example project is quite unambitious but I hope that it shows that Nix provides a wealth of possibilities in the WebAssembly domain (and other domains like it). Wasm is not trivial to build and package and Nix is a far better tool for it than the usual Makefiles and Bash. From a deployment perspective, Nix is typically seen as a tool that can build things like OCI containers or artifacts for running virtual machines (like [ISOs]). But the case of Wasm shows that Nix would be indispensable even in some future world where our industry has gone all-in on Wasm and moved beyond both containers and VMs. [app]: https://github.com/DeterminateSystems/nix-wasm-example/tree/main/src/main.rs [build-wasi]: https://github.com/DeterminateSystems/nix-wasm-example/blob/c69bd6eff6ba25ff9ba887d6688538c5279b4252/flake.nix#L97-L114 [build-wasm-pkg]: https://github.com/DeterminateSystems/nix-wasm-example/blob/c69bd6eff6ba25ff9ba887d6688538c5279b4252/flake.nix#L156-L180 [build-wasmedge-exec]: https://github.com/DeterminateSystems/nix-wasm-example/blob/c69bd6eff6ba25ff9ba887d6688538c5279b4252/flake.nix#L136-L153 [build-wasmtime-exec]: https://github.com/DeterminateSystems/nix-wasm-example/blob/c69bd6eff6ba25ff9ba887d6688538c5279b4252/flake.nix#L116-L134 [containers]: https://opencontainers.org [edge]: https://en.wikipedia.org/wiki/Edge_computing [example]: https://github.com/determinateSystems/nix-wasm-example [fermyon]: https://fermyon.com [extensions]: https://www.salaboy.com/2023/04/15/extending-platforms-with-webassembly [isos]: https://docs.fileformat.com/compression/iso [k8s]: https://kubernetes.io [naersk]: https://github.com/nix-community/naersk [nix]: https://zero-to-nix.com/concepts/nix [rust]: https://rust-lang.org [store]: https://zero-to-nix.com/concepts/nix-store [target]: https://blog.rust-lang.org/2024/04/09/updates-to-rusts-wasi-targets.html [vms]: https://www.vmware.com/topics/glossary/content/virtual-machine.html [wabt]: https://github.com/WebAssembly/wabt [wasi]: https://wasi.dev [wasm]: https://webassembly.org [wasm2wat]: https://webassembly.github.io/wabt/doc/wasm2wat.1.html [wasmedge]: https://wasmedge.org [wasmtime]: https://docs.wasmtime.dev [wasm-objdump]: https://webassembly.github.io/wabt/doc/wasm-objdump.1.html [wasm-stats]: https://webassembly.github.io/wabt/doc/wasm-stats.1.html [wasm-strip]: https://webassembly.github.io/wabt/doc/wasm-strip.1.html [wasm-tools]: https://github.com/bytecodealliance/wasm-tools [wat2wasm]: https://webassembly.github.io/wabt/doc/wat2wasm.1.html [wasm-validate]: https://webassembly.github.io/wabt/doc/wasm-validate.1.html [wrapper]: https://nixos.org/manual/nixpkgs/stable/#fun-makeWrapper --- **On community in Nix** Published: April 26, 2024 URL: https://determinate.systems/blog/on-community-in-nix Dear members of the Nix Community, When I created the [Nix](https://nixos.org/) project as part of my PhD thesis research in 2003, I could never have imagined the enthusiasm that people would have for it, the [energetic community](https://nixos.org/community/) that would grow around it, and the myriad of amazing new things that people would use it to help bring to life. I am immeasurably pleased and proud of where the project stands today and extremely optimistic for its longevity and utility as a critical component of the software delivery pipeline for an ever-growing number of developers, teams and businesses. I co-created and assumed leadership of the NixOS Foundation in 2015 out of a sense of duty to further facilitate the rapid growth of the project and the community. The Foundation had modest goals: it is intended to provide continuity for critical assets like the binary cache, and to provide legal and financial support to events like NixCon. It was not intended to “run” the Nix/NixOS community, which has done an admirable job in managing itself without top-down control. I am glad that I helped to establish the Foundation, and that it continues to serve the practical purposes that it should. That said, my participation in the much-needed administrative work that the body does is not my passion and I would be happy to step away from it at some point in the future. In July 2022, I co-founded Determinate Systems, a company that will accelerate innovation and adoption of Nix by simplifying its use and helping users get predictably great results with it. The company, its mission, and my place in its continued development are my focus and passion and are the vehicles through which I am committed to continuing to contribute to the development of Nix and the realization of its goals. In recent months, and more intensely in recent days, a number of issues have been broached by a small group within the community regarding the governance of the project, the foundation, and the community itself. The ongoing debate on these issues has been divisive to the community and deleterious to our collective goals. I would like to provide clarification regarding a number of misperceptions that have been brought to light as a result of the collective discussion. First, I have had rather little involvement in Nixpkgs and NixOS in recent years, having ceded their stewardship to other community members some time ago, and have no more influence over their direction than any other dedicated contributor. Second, I am just one member of the five-member Nix team and hold no more formal authority than the others in determining the direction of the team. Third, I have not been a member of the RFC Steering Committee since new members were elected in January 2024. Fourth, the NixOS Foundation in no way controls or governs the Nix community, which has, since its inception, demonstrated its ability to self-govern well. With regard to the most focal issue raised in recent discussions, that of who should or shouldn't be allowed to sponsor or participate in NixCon, I feel that the chief aim of the Nix project is to give developers the tools they need to do better work, faster. I strongly believe that we should not exclude any company from contributing to, participating in, or utilizing the Nix project in any way. The nature of open source software is such that anyone, working for any company or none, can contribute to and benefit from the work being done by the broader community. That companies create products and technologies that some approve of and others disapprove of is a fact of life. It is my opinion that it is not for us, as open source software developers, to decide whose views are valid and whose are not, and to allow or disallow project or conference participation as a result. We should welcome all contributions that help the project grow and thrive. This is a view that I have and will always hold, regardless of my company's involvement in the project. Regarding alleged conflicts of interest, I want to be clear. My role, participation, and focus on the good work being done at Determinate Systems have been public knowledge since the company's inception. The assertion that Determinate Systems "owns" Nix or seeks to exert out-sized influence over the project, the community, or the foundation is patently false: I am the only member of the Nix team who works for Determinate Systems. Last, I must express my deep disappointment and disbelief at the accusation of excluding people from minority or marginalized backgrounds. As someone who highly values diversity and inclusion, this accusation is not only unfounded but also insulting. Throughout my career, I have consistently supported work towards creating a welcoming and inclusive environment for all individuals, regardless of their background. I continue to support initiatives to ensure that everyone feels valued and appreciated, and I have actively encouraged opportunities to amplify the voices of those who have been historically marginalized. I remain committed to creating a community where everyone feels seen, heard, and valued, and I will not let unfounded accusations detract from this important work. I encourage everyone reading this who feels that they have not been heard or feels displaced to join the Determinate Systems community as we continue working to make Nix as usable and as impactful as possible. Our code of conduct is available [here](https://flakehub.com/policy/code-of-conduct), and you can join our Discord at https://determinate.systems/discord. In closing, I want to thank everyone in the Nix community for their hard work and dedication to the project. I have nothing but respect, admiration, and deep appreciation for all of you and the commitment and progress that you have made. I am excited to see what the future holds for Nix, and I am committed to continuing to do my part to help it grow and thrive. Sincerely, Eelco Dolstra. --- **Introducing FlakeHub Cache** Published: March 13, 2024 URL: https://determinate.systems/blog/flakehub-cache-beta [Binary caching][caching] is truly one of the most wonderful core features of [Nix]. It enables you to fetch the build results of [derivations] rather than building them locally, which makes just about everything you do with Nix—[development environments][devenv], [package builds][packages], continuous integrations runs—so much faster that Nix without caching is essentially a different, lesser tool. But despite our love for Nix caching, we've believed that existing caching solutions are not suitable for secure, production use cases. And so we've opted to build something better. Today, we're excited to announce **FlakeHub Cache**, a powerful new caching concept from us at [Determinate Systems][detsys] that offers [robust per-flake access control](#access-control). FlakeHub Cache is currently in *private beta* and we're actively iterating on the implementation and seeking design partners to try it out. You can sign up for the beta right here and we'll let you know next steps soon: ## The core feature: granular access control \{#access-control} Nix's official binary cache server, [`nix-serve`][nix-serve], has some major gaps that make it unsuitable for secure, production use cases. Most importantly, `nix-serve` serves up a [Nix store][store] as a single monolithic cache, and *everything* in the cache is available to anyone with a public key. This is an unacceptably fast-and-loose access model for most organizations. With FlakeHub Cache's access control model, however, you can grant or deny read and write access at the [flake][flakes] level, which affords you substantially more control. So what does this look like? ### Reading from FlakeHub Cache \{#reading} Let's start with read access to pull from a flake's cache. If your org has a flake called `security-mega-important`, for example, you can provide read access only to a small set of trusted users—or automated agents—in your org. If, on the other hand, you have a flake called `shared` that's meant to be used by everyone, you can provide read access to everyone in your org. ### Writing to FlakeHub Cache \{#writing} In terms of write access to push to a flake's cache, we've opted for a model centered around controlled, authenticated build environments. FlakeHub Cache currently allows pushing to the cache [solely](#model) as part of [GitHub Actions][actions] runs (with [GitLab] support on the way). This means that there is *no ad-hoc push access whatsoever*, so you can't accidentally push to a cache in a shell script or CLI command. The granular and controlled nature of this model is far more suited for organizations with demanding security and other requirements, and a significant leap beyond existing solutions. ### Authentication model \{#auth} In order to grant or deny access, of course, FlakeHub Cache needs to figure out who—or what—is making a request. Authentication for FlakeHub Cache is based on [JSON Web Tokens][jwt] (JWTs). We currently use [GitHub] as our JWT authentication provider but will be adding [GitLab] support soon. In the future, this flexible model will enable us to support a wide variety of authentication solutions, including [SAML] and various [Single Sign-on][sso] (SSO) providers. Static tokens are great for some use cases—and FlakeHub enables you to create such tokens in the UI—but we're opting to go beyond this model. ## How the cache itself works \{#how} FlakeHub Cache is able to provide [granular access control](#access-control) because of instead of serving an entire [Nix store][store] as "the cache," FlakeHub Cache applies a **slice** abstraction, where a slice is some subset of all store paths. With FlakeHub Cache enabled, each flake you publish on [FlakeHub] gets its own slice and read and write access is applied at this level. When a user has [authenticated](#auth) with FlakeHub Cache, it determines which slices you're allowed to access and combines those slices into a single **view** of the cache. With this view abstraction, FlakeHub Cache can make decisions like these: * User `devops_aficionado_123` from the `WidgetsDotCom` org *may* pull from the cache for the `WidgetsDotCom/devops` flake. * User `AnaBooper` from the `WidgetsDotCom` org *may not* pull from `WidgetsDotCom/super-secure` flake. Although these users are in the same organization, WidgetsDotCom can make access decisions on a per-flake basis and thus choose whichever access patterns and collaboration models they wish. This model is certainly more robust from a security standpoint but it also: 1. **Is much simpler**. When using Nix, you only need to configure FlakeHub Cache; there's no need to distribute public keys for a multitude of them. 1. **Offers better performance**. Using only one platform for caching means fewer network round trips and fewer authentication handshakes. In addition, we run FlakeHub Cache in a variety of regions with CDN-backed storage. This results in a clear speed-up in your Nix builds. In addition to access control, things like garbage collection are also configurable at the flake level. ## A new caching model \{#model} Two core aspects of using FlakeHub Cache make it a significant departure from other available cache systems: 1. **No public caches**. That's right: FlakeHub Cache doesn't allow you to expose public caches like [cache.nixos.org][cno]. You can pull from the cache for a specific flake only if you're (a) authenticated and (b) have [permission](#access-control). That means that you can't even *accidentally* make caches for specific flakes truly public. We're open to exploring public caches in the future but would only do so with due care. 1. **Push only from CI**. At the moment, when you create FlakeHub Cache authentication tokens, those only apply to *pulling* from the cache. You can't *push* to the cache in an ad-hoc way using a CLI or other tool; you can only push from [GitHub Actions][actions], particularly the [Magic Nix Cache Action][magic-action] (with [GitLab support coming soon][gitlab]). If you enroll in the beta, you can enable pushing with this one-liner in your Actions configuration: ```yaml permissions: contents: read id-token: write steps: - uses: actions/checkout@v3 - uses: DeterminateSystems/determinate-nix-action@v3 - uses: DeterminateSystems/magic-nix-cache-action@main - run: nix flake check ``` With that in place, anything that you `nix build` is automatically pushed to the cache—if authorized to!—without any need for separate push commands. ## A bold step forward We firmly believe that FlakeHub Cache is the first enterprise-ready cache in the Nix ecosystem. Public caches and caches without granular access control are not suited for a broad range of use cases and even whole industries, particularly those subject to exacting compliance standards, regulatory regimes, and security requirements. If you're compelled by the power of Nix but have concerns about existing solutions, this might be precisely the sea change you've been waiting for. On top of that, Determinate Systems is the only [SOC2]-compliant vendor in the Nix ecosystem, which should make otherwise-difficult discussions with the higher-ups go much more smoothly. You can try out FlakeHub Cache for yourself soon. Sign up below, configure one of your GitHub Actions runs to use the Magic Nix Cache Action, make a small update to your Nix configuration, and you'll experience this sea change first hand. [actions]: https://docs.github.com/en/actions [caching]: https://zero-to-nix.com/concepts/caching [cno]: http://cache.nixos.org [derivations]: https://zero-to-nix.com/concepts/derivations [detsys]: / [devenv]: https://zero-to-nix.com/concepts/dev-env [flakehub]: https://flakehub.com [flakes]: https://zero-to-nix.com/concepts/flakes [github]: https://github.com [gitlab]: https://gitlab.com [jwt]: https://jwt.io [magic-action]: https://github.com/DeterminateSystems/magic-nix-cache-action [nix]: https://zero-to-nix.com [nix-serve]: https://nixos.org/manual/nix/stable/package-management/binary-cache-substituter [packages]: https://zero-to-nix.com/concepts/packages [saml]: https://en.wikipedia.org/wiki/SAML_2.0 [soc2]: https://www.imperva.com/learn/data-security/soc-2-compliance/ [sso]: https://en.wikipedia.org/wiki/Single_sign-on [store]: https://zero-to-nix.com/concepts/nix-store --- **KVM on GitHub Actions** Published: November 29, 2023 URL: https://determinate.systems/blog/kvm-on-github-actions [KVM] is the most widely used virtualization framework for Linux due to its close integration with the Linux kernel. It's become a mainstay in the [Nix] community because NixOS's innovative [test framework][tests] relies on it for virtualization. I'm pleased to announce that last week the [Determinate Nix Installer Action][action] began enabling KVM on Linux [GitHub Actions runners][actions] by default. Add one line of code to your YAML Actions configuration and you're good to go: ```yaml - uses: DeterminateSystems/determinate-nix-action@v3 ``` ## What has changed Despite KVM's popularity, Nix folks have been essentially blocked from using KVM in [GitHub Actions][actions]. This is a shame for many reason, but above all because KVM is [required] for running [NixOS tests][tests]. This means that a major piece of Nix and NixOS has required solutions like custom runners—which is fine but often a pretty heavy lift. And beyond NixOS tests, this also unblocks fun VM-related things like running [Firecracker] VMs in Actions. ## Requirements Please note that GitHub's policies stipulate that KVM is only available on larger, paid Actions runners. Our testing reveals, however, that they generously provide it to public projects as well. ## How we did it Well, it turns out that Linux GitHub Actions runners _do_ support KVM. They just need a little coaxing (in the form of a few setup commands). You can see how we do it in [this pull request][pr]. ## Disabling KVM As I mentioned above, the baseline GitHub Actions configuration for the Determinate Nix Installer will automatically enable KVM: ```yaml - uses: DeterminateSystems/determinate-nix-action@v3 ``` You can, however, disable this behavior if you need to: ```yaml - uses: DeterminateSystems/determinate-nix-action@v3 with: kvm: false ``` ## Conclusion This is indeed a small change but one that I believe could have a major impact in the Nix community. Thorough testing is crucial for using [NixOS] in production environments and I'm relieved to see this barrier to entry removed. If you have other ideas for how we can improve the experience of using Nix on GitHub Actions—or more broadly—please [get in touch][support] and let us know how! [action]: https://github.com/DeterminateSystems/determinate-nix-action [actions]: https://docs.github.com/actions [firecracker]: https://firecracker-microvm.github.io/ [kvm]: https://linux-kvm.org/page/Main_Page [nix]: https://zero-to-nix.com/concepts/nix [nixos]: https://zero-to-nix.com/concepts/nixos [pr]: https://github.com/DeterminateSystems/determinate-nix-action/pull/56 [required]: https://github.com/NixOS/nixpkgs/blob/bd9c192fc0715fce7b28d33c41f854c5219c2de8/nixos/lib/testing/run.nix#L44 [support]: mailto:support@flakehub.com [tests]: https://hydra.nixos.org/project/nixos --- **A graphical installer for Nix** Published: November 24, 2023 URL: https://determinate.systems/blog/graphical-nix-installer > Update, 2024-11-08: Determinate Nix's graphical installer has replaced this post's installer. > The original graphical installer discussed here is no longer maintained, and the links now point to the Determinate Nix package. Here at Determinate Systems, our core goal is to [make Nix better][better] along as many axes as we possibly can. That has meant building major platforms geared toward Nix users, like [FlakeHub]. But sometimes smaller interventions can make a big difference and today I'm eager to tell you about one: a **graphical installer** for Nix users on macOS. As Nix folks—and especially the Nix curious—know all too well, getting Nix to work on a machine in the first place is _not_ trivial. We built the [Determinate Nix Installer][dni] to make this as painless as possible on Linux, macOS, and Windows Subsystem for Linux (WSL). Invoke just one command and you're off to the races with a next-generation installer written in [Rust]: ```shell curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install ``` But this week we decided to take things even further and offer a **one-click install** option for macOS. Give it a try right now: Don't worry: if you already have Nix installed, this won't harm your system, as we've built the Determinate Nix Installer to gracefully handle previous Nix installs. ## Why a graphical installer? This is a great question because the standard Determinate Nix Installer requires just a single command. Why not stop there? Two reasons: 1. The package is [signed] by Determinate Systems and checked for malware. This means that the chain of cryptographically validated trust is extended further than before and that you can be confident that the installer arrived intact (as in: not tampered with). We have more work to do here but this is a great first step. 1. The package is readily usable by [**mobile device management**][mdm] (MDM) platforms like [Mosyle]. This can serve as a powerful way to distribute Nix to users in managed environments—without needing to run a shell command themselves. ## Future plans Nix users on the Determinate Systems [Discord server][discord] have responded quite positively to the graphical installer thus far. But we're not done just yet. We're currently working on two things: 1. Providing ability to customize the installation. Unfortunately, you can't provide custom parameters for a given `.pkg` installer, which means you can't tweak the knobs of your Nix installation _during_ the installation process itself. But what you _can_ do is generate custom `.pkg` files, and we're currently working on ways to take user preferences and generate them on the fly. 1. Expanding the range of what the Determinate Nix Installer—via the graphical installer—can achieve. That means going beyond installing Nix to installing Nix-related tools like [Home Manager][hm] and [nix-darwin]. Imagine clicking a button and getting a whole suite of Nix tools ready to go—or clicking nothing and letting an MDM platform do the work. If you're on macOS, give the graphical installer a try and let us know how it goes on [Discord]. We look forward to hearing from you. [better]: /blog/we-want-to-make-nix-better [discord]: https://discord.gg/invite/a4EcQQ8STr [dni]: https://github.com/DeterminateSystems/nix-installer [flakehub]: https://flakehub.com [hm]: https://github.com/nix-community/home-manager [mdm]: https://en.wikipedia.org/wiki/Mobile_device_management [mosyle]: https://mosyle.com [nix-darwin]: https://github.com/LnL7/nix-darwin [rust]: https://rust-lang.org [signed]: https://developer.apple.com/developer-id --- **Lessons from 1 million Nix Installs** Published: November 7, 2023 URL: https://determinate.systems/blog/lessons-from-1-million-nix-installs Nix has an [official installer][installer] that I feel has served the community reasonably well over the years. From the beginning it had some issues that made me long for a better alternative—but not so much that I considered rethinking the installer from the ground up. That changed about a year ago. I was struggling with [the installer's Bash scripts][tmpdir] and trying to handle an edge case around temporary directories. Fixing this bug took _nearly a week_, and the end result? _Five_ lines of code—and even when we were done we still weren't sure if it was right. At that moment, I knew that we at Determinate Systems needed to start with a clean slate and build something great, not in Bash but rather in the much more robust and expressive [Rust]. From the beginning we focused on reliability, user experience, and providing a modern Nix experience out of the box. And I firmly believe that we have succeeded. The [Determinate Nix Installer][dni], as we came to call it: - **enables flakes by default.** Nix users both recent and long term have overwhelmingly adopted [flakes]. Our installer always enables flakes and offers a stability guarantee that any flakes that work today will work tomorrow. Users of the Determinate Nix Installer don't need to hesitate or worry about adopting flakes. - **has an uninstaller.** Don't like Nix? No worries. Our installer has a single-command, safe, and thorough uninstaller. - **works almost anywhere.** Install to Linux, macOS, or WSL, on a Steam Deck, inside a Docker container, on an OSTree distro, you name it. If it doesn't work, it's a bug! - **has a [GitHub Action][action].** Add `uses: DeterminateSystems/determinate-nix-action@v3` to your GitHub Actions workflow and you're done. You'll get a safe and stable installation of Nix, with flakes enabled, and all the predictability you're looking for in CI. - **survives macOS upgrades.** This might seem a bit funny-sheesh, but it is true. We [wrote a whole post][posts-nix-survival-mode-on-macos] on it—so check it out. - **isn't scary.** Many folks over the years have uninstalled Nix because the installer was scary. The Determinate Nix Installer is different. Its output is succinct, clear, and feels safe. A user's first introduction is important and we leave the user feeling good about installing Nix. So just _how_ successful has the Determinate Nix Installer been? Today I'm pleased to announce that it has successfully installed Nix **over one million times** since we first [introduced it in January of 2022][posts-determinate-nix-installer]. ## Targeting 100% success We strive for a 100% success rate when installing Nix. That doesn't mean that we put the files in the right place and call it good. Nix has to _work_ the way that users expect it to. A "successful failure" is still a failure in our eyes. We're not big fans of software "phoning home." Nobody loves it, and every change we make to our diagnostics receives careful reviews and strong critique on principle. At the same time, we could not make the most reliable installer without it. The Determinate Nix Installer collects a minimal amount of diagnostic data after every installation. This data includes the OS and architecture of the computer, whether the install succeeded, and some sanitized amount of failure information, such which part of the installation process failed. Collecting this data is a critical component of improving the installer and targeting the most important problems that users are facing. Reaching 100% may not be possible but we have the results to prove we're striving to give users the best experience every time. Overall, we're tracking approximately a **99.4%** success rate. ## Rolling deployments One important way we're able to retain our success rate is through our carefully orchestrated, rolling deployments. We don't just flip a switch and move 100% of our users to new releases all at once. This is risky and it doesn't treat users with the consideration they deserve. Nix and the Determinate Nix Installer are load-bearing components of our users' stacks and we have to respect that. Our releases start by rolling out to only 20% of requests from GitHub Actions. We start with GitHub Actions because the environment is ephemeral and failure cases can be resolved by restarting the job. This means that users on long-term devices don't get a bad experience and CI users are likely to hit "re-run" when they encounter a weird edge case. We carefully track the new release and monitor to see if users are experiencing an increase in installation failures. But we don't stop there. Like I mentioned earlier, our goal is not just to throw Nix down on the host and call it good. Our goal is to deliver a working version of Nix that doesn't break users' setups. To accomplish this, [our GitHub Action][action] reports back anonymized summary data for public GitHub Actions workflows. The data is a little bit noisy but it is also valuable, and in practice we find that the rate of workflow failures is consistent between two releases unless Nix or the installer is broken. This data along with some diagnostics data enables us to identify problems and regressions in Nix itself for real users in a way that nobody else in the Nix landscape is doing. Over time, we carefully ramp up the GitHub Actions installations until we reach 100%, and alongside that we also ramp up the upgrade for users outside of CI. Because our failure rate in CI is so low, we're able to take careful, measured steps to roll out new features and updates without big-bang releases that can spoil many an afternoon. The long tail of error conditions is long but I do believe that our results speak for themselves. ## Lessons learned on our way to 100% User machines in particular have a uniquely _fascinating_ history (to put it diplomatically). ### macOS' security model is robust Installing Nix on macOS means reckoning with an ever-tightening security model. This is generally good for users but it means that Nix and its installer have to constantly keep up. Our installer is written in Rust, which means that adapting to the continuous upstream changes is safer; we're not fighting the language as the official installer must do with Bash. Our installer (and uninstaller!) successfully navigates configuring `synthetic.conf` and `fstab` and also creating APFS volumes—with encryption, and more. Using Rust has been crucial to doing things the right way. We're also able to do experiments and make improvements like switching from named APFS volumes to using UUIDs, which enables us to solve tricky problems surrounding [systems that boot without the Nix store mounted][nix-installer-issue-212]. ### MUSL builds and nscd/sssd on Linux For portability across Linux distributions, the Determinate Nix Installer is statically compiled using [MUSL]. In Rust, this means targeting `x86_64-unknown-linux-musl` and the `crt-static` target feature as documented in the [Rust Reference][rust-static-dynamic-runtime]. This unfortunately brings a new set of unique issues. During one step of the installation process we use [`nix::unistd::User`][`nix::unistd::User`], which uses the [`getpwnam_r`][`libc::getpwnam_r`] syscall. We've received [several reports][nix-installer-issues-nscd] that indicate that programs like `nscd` and `sssd` can override the `getpwnam_r` syscall. In these situations, one workaround is to build our installer yourself with `cargo install nix-installer` and run that. But we don't love this solution and we hope to provide something more compelling, such as creating [glibc]-based release binaries. ### Creating and deleting users Serially creating ~32 users for Nix takes an annoyingly long time, measuring _seconds_ per user on some machines. Early in development we experimented with creating users in parallel to speed up the process. This turned out to be problematic on Mac and Linux due to locking and other parallelism-related issues. We also looked at directly editing `/etc/passwd` and other files, but we are concerned this may cause further issues in enterprise environments with central user directories. In addition, we adopted the [`auto-allocate-uids`][`auto-allocate-uids`] feature from Nix, which did make installation much faster but caused other issues. On macOS, for example, we experienced problems building Nix (of all things) because `whoami` no longer worked. We had problems on Linux, too. In [issue #539][nix-installer-issue-539] we noticed that some distributions experienced errors like `setting uid: invalid argument`. We ultimately rolled the feature back but one day we'd love to find a solution that would let us adopt it again. And the fun continues. In [issue #33][nix-installer-issue-33] we found that deleting users on certain Macs sometimes ends with a permissions error. After quite a bit of investigation, as well as referencing articles like [Can't delete a macOS user with dscl][cant-delete-a-macos-user-with-dscl-resolution] and [When you "can't" delete a user in MacOS][when-you-cant-delete-a-user-in-macos], we uncovered the issue. It seems that you can't delete users on macOS if nobody has logged into the machine graphically. This was a big problem for us since we [run a macOS build farm][mac-build-farm] dedicated to building and testing the installer. We still don't have an automatic fix but we do detect the error and provide instructions on how to resolve it. ### Nix's SSL certificate story needed improvement Issues like [#289][nix-installer-issue-289] and later [#516][nix-installer-issue-516] made it evident that the existing `NIX_SSL_CERT_FILE` environment variable was causing some problems for certain installations, as well as confusion in some users. Running `nix build`, for example, would sometimes produce errors like this: ``` warning: error: unable to download '...': SSL peer certificate or SSH remote key was not OK (60); retrying in 337 ms ``` The problem appeared to be stem from inconsistencies in how `NIX_SSL_CERT_FILE` was being handled. During discussions with Eelco we concluded that the best solution would be to lift the `NIX_SSL_CERT_FILE` into a configuration option inside users' `nix.conf` [nixos/nix#8062][nix-pr-8062] configuration files. This appears to have solved most of the issues we were seeing. ### Uninstallation order is important A recurring issue that cropped up on our issue boards was a positively _bizarre_ CA certificate issue on Macs characterized by [pull request #608][nix-installer-pr-608]. Our first few reports made little sense. Why was Nix trying to access `/etc/ssl/certs/ca-certificates.crt`? That path doesn't normally exist on Mac and the install process doesn't involve it! [Reproducing the issue][nix-installer-pr-528-comment-1629394069] required these steps: 1. Install Nix 2. Install [`nix-darwin`][nix-darwin] 3. Uninstall Nix either with `/nix/nix-installer uninstall` or the [official guide][nix-reference-uninstall] 4. Reinstall Nix Uninstalling Nix before uninstalling `nix-darwin` leaves a Launch Daemon called `org.nixos.activate-system`. Leaving this Launch Daemon lingering causes issues with the `NIX_SSL_CERT_FILE` environment variable, which in turn spoils reinstalls. This issue prompted us to add new pre-install and pre-uninstall checks to warn about the issue before you hit it. There is a [workaround][nix-installer-pr-608] and we hope that a future release of the installer will provide a robust cure for this issue. ### Containers are complicated There are several popular container runtimes that differ in subtle ways. Installation is pretty normal when targeting a [Podman] container with [systemd], for example, but Docker containers can't run systemd, which complicates the installation. In some runtimes, Nix's sandboxing isn't a viable option due to highly restrictive sandboxing of the container itself. At one point, we made a matrix of different options that worked for Podman and Docker but the complexity got the best of us. In the end, we found [two configurations][nix-installer-readme-containers] that worked in the most common use cases. It feels like there is still A story to be told on this particular issue and we'd be glad to find a better solution. ## Thank you to our collaborators Whether it's a carefully described issue, a drive-by pull request, or even seeing your friendly faces at the [Installer Working Group][nix-discourse-installer-wg] meetings, we want to say **thank you** 🎉😊 for collaborating with us on this project. It's been extremely uplifting to be able to participate in these greater community discussions. We continue to hope that the upstream project adopts the Determinate Nix Installer for itself. In particular, a big thank you to [Abathur][nix-discourse-user-abathur] and [Mkenigs][nix-discourse-user-mkenigs] for continued in-depth collaboration. ## What's next? An early design choice was that our installer should have a public API for building custom installers on top of it. Ultimately, this hasn't received a lot of interest and has made creating the user experience we want much more complicated. We haven't yet decided if we want to keep this API, but if this is important to you, please let us know on our [Discord][discord-invite]. We're well on our way to our next million installations, but before we get there it'd be great to call it 1.0.0. If you'd like to chat about Nix and get help with flakes, please join us on our flake-forward [Discord][discord-invite]! [action]: https://github.com/DeterminateSystems/determinate-nix-action [cant-delete-a-macos-user-with-dscl-resolution]: https://web.archive.org/web/20240422065844/https://it.megocollector.com/macos/cant-delete-a-macos-user-with-dscl-resolution/ [discord-invite]: https://discord.gg/bU9enxBwJt [dni]: https://github.com/DeterminateSystems/nix-installer [flakes]: https://zero-to-nix.com/concepts/flakes [installer]: https://nixos.org/download [glibc]: https://www.gnu.org/software/libc [mac-build-farm]: https://buildkite.com/blog/ephemeral-mac-os-builds-buildkite-nix-tailscale [musl]: https://www.musl-libc.org [nix-darwin]: https://github.com/LnL7/nix-darwin [nix-discourse-installer-wg]: https://discourse.nixos.org/t/nix-installer-workgroup/21495 [nix-discourse-user-abathur]: https://discourse.nixos.org/u/abathur/summary [nix-discourse-user-mkenigs]: https://discourse.nixos.org/u/mkenigs/summary [nix-installer-issue-212]: https://github.com/DeterminateSystems/nix-installer/issues/212 [nix-installer-issue-33]: https://github.com/DeterminateSystems/nix-installer/issues/33 [nix-installer-issue-289]: https://github.com/DeterminateSystems/nix-installer/issues/289 [nix-installer-issue-516]: https://github.com/DeterminateSystems/nix-installer/issues/516 [nix-installer-issue-539]: https://github.com/DeterminateSystems/nix-installer/issues/539 [nix-installer-issues-nscd]: https://github.com/DeterminateSystems/nix-installer/issues?q=nscd [nix-installer-pr-528-comment-1629394069]: https://github.com/DeterminateSystems/nix-installer/issues/528#issuecomment-1629394069 [nix-installer-pr-608]: https://github.com/DeterminateSystems/nix-installer/issues/608 [nix-installer-readme-containers]: https://github.com/DeterminateSystems/nix-installer#in-a-container [nix-pr-8062]: https://github.com/NixOS/nix/pull/8062 [nix-reference-uninstall]: https://nixos.org/manual/nix/stable/installation/uninstall [podman]: https://podman.io [posts-nix-survival-mode-on-macos]: /blog/nix-survival-mode-on-macos [posts-determinate-nix-installer]: /blog/determinate-nix-installer [rust]: https://rust-lang.org [rust-static-dynamic-runtime]: https://doc.rust-lang.org/stable/reference/linkage.html#static-and-dynamic-c-runtimes [systemd]: https://systemd.io [tmpdir]: https://github.com/NixOS/nix/pull/6916 [when-you-cant-delete-a-user-in-macos]: https://web.archive.org/web/20240422065842/https://www.aixperts.co.uk/?p=214 [`auto-allocate-uids`]: https://nixos.org/manual/nix/stable/contributing/experimental-features#xp-feature-auto-allocate-uids [`libc::getpwnam_r`]: https://docs.rs/libc/latest/libc/fn.getpwnam_r.html [`nix::unistd::User`]: https://docs.rs/nix/latest/nix/unistd/struct.User.html --- **Nix Survival Mode: sheltering Nix from macOS upgrades** Published: October 25, 2023 URL: https://determinate.systems/blog/nix-survival-mode-on-macos Tens of thousands of macOS users install Nix using the [Determinate Nix Installer][determinate-nix-installer], and I'm one of them. Last February, [we introduced the Determinate Nix Installer][dni-blog-post] to solve a multitude of reliability problems and failure conditions. This has been a rousing success, but we always aspire to reduce friction and improve the user experience. Our goal is 100% success with zero fuss. Despite a rounding error of install-time failures, hilariously, most Nix installations are broken after macOS upgrades. Not great: this is not the user experience we're going for! People usually like software better when it stays installed, and you'd think Nix—striving for predictability—wouldn't do this. Well, good news: **Nix installations from the Determinate Nix Installer after version v0.14 survive macOS upgrades.** 🎉 The latest release of our installer now creates a global Launch Daemon which restores the necessary pieces of Nix's installation at boot time. That means the Nix you installed stays installed, eliminating the largest source of friction for macOS users and demonstrating our commitment to being the most reliable, resilient, and safest way to install Nix. Try it on your Mac today: ```shell curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install ``` Or integrate the most reliable Nix installer in GitHub Actions: ```yaml - uses: DeterminateSystems/determinate-nix-action@v3 ``` [determinate-nix-installer]: https://github.com/DeterminateSystems/nix-installer [dni-blog-post]: /blog/determinate-nix-installer --- **FlakeHub updates: lightning-fast mirroring and webhook integrations** Published: October 6, 2023 URL: https://determinate.systems/blog/flakehub-updates We're a [little over a month][fh-post] into the life of [FlakeHub] and it's been quite the journey so far. Well over 150 flakes have been published by almost 100 GitHub orgs and we've been hard at work bringing more and more features to the platform, including [a CLI tool called `fh`][fh] and [flake schemas][schemas]. In this post, I'd like to talk about some recent updates as well as some fun stuff on the way. ## Faster mirroring We've gotten tons of wonderful feedback from enthusiastic FlakeHub users on [Discord] and other venues, including a handful of really compelling feature requests. Some FlakeHub users have requested more timely updates to widely used flakes like [Nixpkgs] and we're happy to say that we now update all flakes **within 60 seconds of an upstream release**. _[Mirrored]_ flakes are available on FlakeHub but haven't yet been published by the original authors. Mirroring enabled us at Determinate Systems to provide widely used flakes to users upon launch. Initially, we published Nixpkgs twice a day, which meant that people relying on it could see up to a 12-hour lag between update and delivery. But user feedback has convinced us that this isn't nearly frequent enough, so we've implemented a long-running process that listens for pushes to Nixpkgs and immediately publishes them to FlakeHub. While we do want upstream projects to publish directly to FlakeHub, we hope that this will tide over FlakeHub users (and us at DetSys!) in the meantime. ## Beta webhooks and integrations FlakeHub users can now **subscribe to webhooks for any published flake**. Users have been asking for pub/sub notifications for updates to projects like Nixpkgs, and FlakeHub is now poised to deliver. Some use cases for this include triggering [`flake.lock` updates][updates] or an internal rebase for a fork. This feature is currently in **limited beta**, so contact [support@flakehub.com][support] if you'd like to be an early tester. [Keep an eye out][blog], we have more on its way! [blog]: / [discord]: https://determinate.systems/discord [fh]: /blog/flakehub-cli [fh-post]: /blog/introducing-flakehub [flakehub]: https://flakehub.com [mirrored]: https://docs.determinate.systems/flakehub/concepts/mirroring [nixpkgs]: https://github.com/NixOS/nixpkgs [schemas]: /blog/flake-schemas [support]: mailto:support@flakehub.com [updates]: https://github.com/marketplace/actions/update-nix-flake-lock --- **Creating and modifying flakes using the FlakeHub CLI** Published: September 20, 2023 URL: https://determinate.systems/blog/fh-updates Last month we at [Determinate Systems](/) announced the [initial release][flakehub-post] of [FlakeHub], a brand new platform for publishing and exploring [Nix flakes][flakes], with features like [SemVer] for flakes and robust [search]. Soon thereafter, we released [`fh`][fh], a CLI tool for interacting with FlakeHub. The initial release of `fh` offered a modest set of capabilities, including the ability to [list][fh-list] and [search][fh-search] all publicly listed flakes. But recently we've been iterating on the tool in earnest and today we'd like to announce two new features: [`fh add`](#fh-add) and [`fh init`](#fh-init), both of which substantially improve the ergonomics of working with [`flake.nix` files][flakes]. Try out `fh` right now: ```shell nix run "https://flakehub.com/f/DeterminateSystems/fh/0.1.5.tar.gz" ``` See the [README instructions][install] to install it on your system. ## `fh add` The [`fh add`][fh-add] command enables you to quickly add new [flake inputs][inputs] to an existing flake. This command, for example, would add [Nixpkgs] to your `flake.nix`: ```shell fh add NixOS/nixpkgs ``` If you specify a flake reference as `:org/:project`, as in this example, `fh` infers that you mean a [FlakeHub] reference and searches to ensure the flake exists. It then adds a reference to the latest version of the flake. The resulting `inputs` block may look something like this (notice the FlakeHub address and the [specific version][flakehub-versions]): ```nix { inputs = { nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.490944.tar.gz"; # Other inputs }; } ``` You can also use `fh add` to add non-FlakeHub references: ```shell fh add github:DeterminateSystems/nuenv ``` ## `fh init` Though it's nice to have a convenient way to update [flakes] that already work, creating new flakes from scratch can be a bit of a chore. I've done it enough times that it's now ingrained in my muscle memory but I don't look forward to it and we at Determinate Systems suspect that _no one_ does. So we added an [`fh init`][fh-init] command to the FlakeHub CLI. All you have to do is run it in the root of a project and 🪄🦄🧚‍♀️ magical things happen 🪄🦄🧚‍♀️: ```shell cd /path/to/your/project fh init ``` `fh init` assumes that you don't yet have a `flake.nix` at the root of your project (and if you do, it only overwrites it if you explicitly opt for that). Then it looks at the contents of your project and asks you a series of questions about what you want. At the end of that process, it outputs a `flake.nix` file that you can immediately use in your project. That's it! Try it in a flake-less project now: ```shell nix run "https://flakehub.com/f/DeterminateSystems/fh/0.1.5.tar.gz" -- init ``` Here's an example `fh init` flow: - If your project has a `Cargo.toml` file, it asks if this is indeed a [Rust] project. - If you say yes, then it checks for a [`rust-toolchain`][rust-toolchain] or [`rust-toolchain.toml`][rust-toolchain] file. - If one is present, it uses that to create your Rust environment; if not, it uses the latest stable version of Rust. [Similar flows][handlers] are currently available for Go, Java, JavaScript, PHP, Python, Ruby, and Zig. Beyond these language-specific helpers, `fh init` also enables you to: - Provide a description of the flake - Include helpful doc comments explaining various aspects of the flake - Include common utilities like [curl], [jq], [Git], and [nixpkgs-fmt] (the Nix formatter that we prefer at Determinate Systems) - Generate a flake-friendly [direnv] configuration file - Select which systems you want the flake to support (`x86_64-linux`, `aarch64-darwin`, etc.) - Provide custom environment variables As we usually do at DeterminateSystems, we have strong—though hopefully informed—opinions about how things should be, and flakes are, [unsurprisingly][stable-flakes], no exception to that. Some opinions that we baked into `fh init`: - Rather than using libraries like [`flake-utils`][flake-utils] or [`flake-parts`][flake-parts], it generates plain old Nix functions as helpers for generating [per-system outputs][system-specificity]. - It always adds [Nixpkgs] as an input, as it's necessary for pretty much all the flakes that `fh init` generates. It does, however, enable you to select which version of Nixpkgs you want. But `fh init` is just a starter. You're always free to customize at will; after all, it's just Nix code! We should also note here that `fh init` is different from [flake templates][templates] and the related [`nix flake init`][nix-flake-init] command. Flake templates essentially copy pre-existing files—including `flake.nix` files if you want—into your current directory (or a new directory). `fh init`, conversely, takes the specifics of your project and your desires into account. ## More on the way for `fh` \{#more} We're confident that `fh add` and `fh init` will polish off some of the rough edges of working with flakes. But we have plans to improve `fh init` pretty dramatically in the near future, including: - Support for generating [flake outputs][outputs] beyond [`devShells`][dev-env], such as package outputs - More language- and tool-specific interactive flows In the meantime, please don't hesitate to provide feedback in the form of [issues] and [pull requests][prs]. Or join us on [Discord] if you just want to chat about [FlakeHub]. [curl]: https://curl.se [dev-env]: https://zero-to-nix.com/concepts/dev-env [direnv]: https://direnv.net [discord]: https://discord.gg/invite/a4EcQQ8STr [fh]: /blog/flakehub-cli [fh-add]: https://github.com/determinateSystems/fh#add-a-flake-published-to-flakehub-to-your-flakenix [fh-init]: https://github.com/DeterminateSystems/fh#initialize-a-new-flakenix-from-scratch [fh-list]: https://github.com/DeterminateSystems/fh#listing-flakes-organizations-and-versions [fh-search]: https://github.com/DeterminateSystems/fh#searching-published-flakes [flake-parts]: https://github.com/hercules-ci/flake-parts [flake-utils]: https://flakehub.com/flake/numtide/flake-utils [flakehub]: https://flakehub.com [flakehub-post]: /blog/introducing-flakehub [flakehub-versions]: https://docs.determinate.systems/flakehub/concepts/semver [flakes]: https://zero-to-nix.com/concepts/flakes [git]: https://git-scm.com [handlers]: https://github.com/DeterminateSystems/fh/tree/main/src/cli/cmd/init/handlers [inputs]: https://zero-to-nix.com/concepts/flakes/#inputs [install]: https://github.com/DeterminateSystems/fh#installation [issues]: https://github.com/determinateSystems/fh/issues [jq]: https://jqlang.github.io/jq [nix-flake-init]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-init [nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs [nixpkgs-fmt]: https://github.com/nix-community/nixpkgs-fmt [outputs]: https://zero-to-nix.com/concepts/flakes/#outputs [prs]: https://github.com/determinateSystems/fh/pulls [rust]: https://rust-lang.org [rust-toolchain]: https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file [search]: https://flakehub.com/search?q=nix [semver]: https://docs.determinate.systems/flakehub/concepts/semver [stable-flakes]: https://discourse.nixos.org/t/experimental-does-not-mean-unstable-detsyss-perspective-on-nix-flakes/32703 [system-specificity]: https://zero-to-nix.com/concepts/system-specificity [templates]: https://zero-to-nix.com/concepts/flakes/#templates --- **Experimental does not mean unstable** Published: September 6, 2023 URL: https://determinate.systems/blog/experimental-does-not-mean-unstable Although Nix flakes are currently marked as experimental, the on-the-ground experience of actually _using_ flakes has been quite stable since their initial release in November 2021. Even the so-called "unified CLI," also marked as experimental, has seen remarkably few breaking changes in that time. Evidence from GitHub strongly suggests that the Nix community is moving in the direction of flakes and the unified CLI _en-masse_. And I hear from more recent Nix adopters over and over again that flakes were crucial to their learning journey. I believe that flakes are stable, and I'm not alone. My position is that the experimental flag should have been removed ages ago—or never introduced in the first place. I don't think that flakes need a lengthy stabilization process. Nix users of all stripes have been using flakes in production for years and to great effect. The "experimental" label needs to be removed. Flakes are a proven technology and deserve to be officially recognized as such. Nonetheless, flakes in their current form have detractors, and those detractors make some reasonable points. I want to address three common criticisms of flakes. 1. **Source explosion**. Many of you have probably seen the article about [1,000 instances of Nixpkgs](https://zimbatm.com/notes/1000-instances-of-nixpkgs). It's a good read but I disagree with the conclusions. With evaluation caching and the functional nature of Nixpkgs, source explosion is generally mitigated using the `follows` mechanism built into flakes, which reduces the number of unique Nixpkgs instances involved in an evaluation. And by embedding further support for version boundaries that [FlakeHub] introduces, the Nix locking mechanism can even make reasonable dependency unification choices automatically. 2. **Cross-compilation**. Some say that cross-compilation using flakes isn't great. Flakes and the unified CLI indeed don't offer specific support for cross-compilation—but the old Nix CLI and channels didn't offer direct support either. Improving cross-compilation in Nix is a large though highly worthy project. But it has nothing to do with flakes and thus no bearing on stabilization. 3. **Lockfile format**. I've seen discussions around this and the terms are always vague. Sometimes, lockfiles can become rather large. I don't have a lot of sympathy for this problem, when Nixpkgs' git history is gigantic and computing is relatively cheap. Nonetheless, this problem is also being worked on and does not need to be solved prior to stabilization, as a new mechanism can increment the `flake.lock` version field. I think that #1 is a fair criticism and an area where I'd like to see forward progress. But I do _not_ see it as a compelling reason to continue marking flakes as experimental. We can declare flakes as stable **today** and find ways to address source explosion under the banner of flakes as a stable feature of Nix. Could flakes be improved? Yes. I do acknowledge that there are some fundamental issues that should be addressed—this is true of just about any technology. But the "experimental" flag should not, in my view, be interpreted as "unstable." It would be reckless to break backwards compatibility in how flakes are evaluated, especially locked flakes. Any changes to flakes should be made with due care and respect for the sake of the thousands of users incorporating Nix into their daily workflows. Determinate Systems is committed to maintaining that compatibility in partnership and in collaboration with the Nix maintenance team. We shouldn't let yet another NixCon, yet another year go by having only a vague sense of what it'll take to get flakes stabilized. It's time to call good enough good enough. I'm calling on the Nix team to remove the experimental flag by the end of the year and to thereby open a new chapter in Nix's history and pave the way for other worthy goals. [flakehub]: https://flakehub.com --- **`fh`: the CLI for FlakeHub** Published: September 1, 2023 URL: https://determinate.systems/blog/flakehub-cli Last week, we released [FlakeHub], a new platform for publishing, discovering, and using [Nix flakes][flakes]. We've been extremely pleased with the initial response: as of this post, more than [60 organizations][fh-orgs] have published a total of over [110 flakes][fh-flakes] to FlakeHub, and many Nix users have already incorporated the platform into their day-to-day workflows. Although we're happy with the web UI for FlakeHub, we know that many workflows will require more programmatic access to the platform. And so today we're announcing the initial release of [`fh`][fh], the CLI for FlakeHub. To give an analogy, `fh` serves the same role for FlakeHub that the [`gh`][gh] CLI serves for GitHub. ## Getting started First off, you'll need Nix 2.17 or above to use `fh`. Check out the [FlakeHub docs][upgrade] for information on how to upgrade. Once you have a compatible version of Nix, there are several ways to get started. You can start a shell session with it installed using [`nix shell`][nix-env-shell]: ```shell nix shell "https://api.flakehub.com/f/DeterminateSystems/fh/*.tar.gz" ``` You can also add it to a Nix [development environment][env] as in this `flake.nix`: ```nix { inputs = { nixpkgs.url = "https://api.flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz"; fh.url = "https://api.flakehub.com/f/DeterminateSystems/fh/0.1.*.tar.gz"; }; outputs = { self, nixpkgs, fh, .. } @ inputs: let supportedSystems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ]; forEachSupportedSystem = f: nixpkgs.lib.genAttrs supportedSystems (system: f { inherit system; pkgs = import nixpkgs { inherit system; }; }); in { devShells = forEachSupportedSystem ({ pkgs, system }: { default = pkgs.mkShell { packages = [ fh.packages.${system}.fh ]; }; }); }; } ``` To install it `fh` in your home environment, we recommend using [Home Manager][hm]. ## Commands ### `fh add` \{#add} `fh add` adds a FlakeHub dependency to a flake. Let's say that you've defined a flake in `flake.nix` in the current directory: ```nix { inputs.nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz"; outputs = { self, nixpkgs }: { # Outputs here }; } ``` This command would add the [`ipetkov/crane`][crane] flake: ```shell fh add ipetkov/crane ``` The resulting `flake.nix`: ```nix { inputs.crane.url = "https://flakehub.com/f/ipetkov/crane/0.13.1.tar.gz"; inputs.nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz"; outputs = { self, nixpkgs, crane }: { # Outputs here }; } ``` `fh add` queries FlakeHub to find the most recent version of that input (in this case [0.13.1][crane-latest] for `ipetkov/crane`). You can also use `fh add` with non-FlakeHub flake references. Some examples: ```shell fh add github:NixOS/patchelf fh ``` ### `fh list` \{#list} `fh list` has three subcommands that are useful for getting a general idea of what's on FlakeHub right now: - `fh list flakes` lists all the publicly listed [flakes][fh-flakes] on FlakeHub - `fh list orgs` lists all the publicly listed [organizations][fh-orgs] on FlakeHub - `fh list releases ` lists all the releases of a flake on FlakeHub. Try `fh list releases tailscale/tailscale`, for example. ### `fh search` \{#search} `fh search` enables you to search FlakeHub's [Algolia] index of flakes using an arbitrary string query. Here's an example: ```shell fh search rust ``` That returns this table: ``` +---------------------------------------------------------------------------------+ | Flake FlakeHub URL | +---------------------------------------------------------------------------------+ | astro/deadnix https://flakehub.com/flake/astro/deadnix | | carlthome/ml-runtimes https://flakehub.com/flake/carlthome/ml-runtimes | | ipetkov/crane https://flakehub.com/flake/ipetkov/crane | | kamadorueda/alejandra https://flakehub.com/flake/kamadorueda/alejandra | | nix-community/fenix https://flakehub.com/flake/nix-community/fenix | | nix-community/lanzaboote https://flakehub.com/flake/nix-community/lanzaboote | | nix-community/nix-init https://flakehub.com/flake/nix-community/nix-init | | nix-community/nixpkgs-fmt https://flakehub.com/flake/nix-community/nixpkgs-fmt | | nix-community/patsh https://flakehub.com/flake/nix-community/patsh | | ryanccn/nyoom https://flakehub.com/flake/ryanccn/nyoom | +---------------------------------------------------------------------------------+ ``` Well, that query returns these results _today_, but this will surely change in the future as more flakes are added, included [Rust]-related flakes. You can also run more complex queries, like these: ```shell fh search "nixos modules" fh search "flake python" ``` ## Going forward `fh` is a modest tool but it brings real improvements to the experience around [Nix flakes][flakes]. It gives you quick access to the steadily expanding constellation of flakes on [FlakeHub], including search, and to programmatically add flakes to your projects. In the near term, we have plans to support interactively creating new `flake.nix` files and to expand `fh`'s ability to modify existing flakes. So keep up to date with the [`fh` flake][fh-flake] for future improvements. [algolia]: https://algolia.com [crane]: https://flakehub.com/flake/ipetkov/crane [crane-latest]: https://flakehub.com/flake/ipetkov/crane/0.13.1 [env]: https://zero-to-nix.com/concepts/dev-env [fh]: https://github.com/DeterminateSystems/fh [fh-flake]: https://flakehub.com/DeterminateSystems/fh [fh-flakes]: https://flakehub.com/flakes [fh-orgs]: https://flakehub.com/orgs [flakes]: https://zero-to-nix.com/concepts/flakes [flakehub]: https://flakehub.com [gh]: https://cli.github.com [hm]: https://nix-community.github.io/home-manager [nix-env-shell]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-env-shell [rust]: https://rust-lang.org [upgrade]: https://docs.determinate.systems/determinate-nix#determinate-nixd-upgrade --- **Flake schemas: making flake outputs extensible** Published: August 31, 2023 URL: https://determinate.systems/blog/flake-schemas [Flakes] are a generic way to package [Nix] artifacts. Flake output attributes are arbitrary Nix values, so they can be [packages], [NixOS modules][modules], [CI jobs][hydra], and so on. While there are a number of "well-known" flake output types that are recognized by tools like the [`nix` CLI][cli]—[`nix develop`][nix-develop], for example, operates on the `devShells` output—nothing prevents you from defining your own flake output types. Unfortunately, such "non-standard" [flake output][output] types have a big problem: tools like [`nix flake show`][nix-flake-show] and [`nix flake check`][nix-flake-check] don't know anything about them, so they can't display or check anything about those outputs. The [`nixpkgs`][nixpkgs] flake, for instance, has a `lib` output that Nix knows nothing about: ```shell # nix flake show nixpkgs github:NixOS/nixpkgs/4ecab3273592f27479a583fb6d975d4aba3486fe ├───... └───lib: unknown ``` This was a problem when we were creating [FlakeHub]: the FlakeHub web interface should be able to display the contents of a flake, including documentation, but we want to do so in an extensible way. Today we're proposing a solution to this problem: **flake schemas**. Flake schemas enable flakes to declare functions that enumerate and check the contents of their outputs. Schemas themselves are defined as a flake output named `schemas`. Tools like `nix flake check` and FlakeHub can then use these schemas to display or check the contents of flakes in a generic way. In this approach, flakes carry their own schema definitions, so you are not dependent on some central registry of schema definitions—_you_ define what your flake outputs are supposed to look like. Here is an example of what the outputs of a flake, extracted using that flake's schemas, look like in FlakeHub: ## Using flake schemas While you can define your own schema definition (see below), usually you would use schema definitions provided by others. We provide a repository named [`flake-schemas`][repo] with schemas for the most widely used flake outputs (the ones for which Nix has built-in support). Declaring what schemas to use is straightforward: you just define a `schemas` output. ```nix { # `flake-schemas` is a flake that provides schemas for commonly used flake outputs, # like `packages` and `devShells`. inputs.flake-schemas.url = github:DeterminateSystems/flake-schemas; # Another flake that provides schemas. inputs.other-schemas.url = ...; outputs = { self, flake-schemas, other-schemas }: { # Tell Nix what schemas to use. schemas = flake-schemas.schemas // other-schemas.schemas; # These flake outputs will now be checked using the schemas above. packages = ...; devShells = ...; }; } ``` ## Defining your own schemas With schemas, we can now teach Nix about the `lib` output mentioned previously. Below is a flake that has a `lib` output. Similar to `lib` in Nixpkgs, it has a nested structure (for example it provides a function `lists.singleton`). The flake also has a `schemas.lib` attribute that tells Nix two things: 1. How to list the contents of the `lib` output. 1. To check that every function name follows the camelCase naming convention. ```nix { outputs = { self }: { schemas.lib = { version = 1; doc = '' The `lib` flake output defines Nix functions. ''; inventory = output: let recurse = attrs: { children = builtins.mapAttrs (attrName: attr: if builtins.isFunction attr then { # Tell `nix flake show` what this is. what = "library function"; # Make `nix flake check` enforce our naming convention. evalChecks.camelCase = builtins.match "^[a-z][a-zA-Z]*$" attrName == []; } else if builtins.isAttrs attr then # Recurse into nested sets of functions. recurse attr else throw "unsupported 'lib' type") attrs; }; in recurse output; }; lib.id = x: x; lib.const = x: y: x; lib.lists.singleton = x: [x]; #lib.ConcatStrings = ...; # disallowed }; } ``` With this schema, `nix flake show` can now show information about `lib`: ```shell # nix flake show git+file:///home/eelco/Determinate/flake-schemas/lib-test └───lib ├───const: library function ├───id: library function └───lists └───singleton: library function ``` While `nix flake check` will now complain if we add a function that violates the naming convention we defined: ```shell # nix flake check warning: Evaluation check 'camelCase' of flake output attribute 'lib.ConcatStrings' failed. ``` ## What schemas are not Flake schemas are not a type system for Nix, since that would be a huge project. They merely provide an _interface_ that enables users to tell Nix how to enumerate and check flake outputs. For instance, for a [NixOS] configuration, this means using the module system to check that the configuration evaluates correctly; for a Nix package, it just means that the output attribute evaluates to a [derivation]. ## Next steps Flake schemas are new, and they're a valuable expansion of the user experience of [FlakeHub] and [Nix flakes][flakes] in general. We believe that incorporating schemas into Nix itself will make flakes more broadly valuable and cover use cases that we haven't yet imagined. We're looking for input from the community to see whether the current schema design covers all use cases. We've submitted a [pull request][pr] to the [Nix project][nix-repo] that adds schema support to [`nix flake show`][nix-flake-show] and [`nix flake check`][nix-flake-check]. Please take a look! As future work, schemas will enable us to finally make flakes _configurable_ in a discoverable way: flake schemas can return the configuration options supported by a flake output—for the `nixosConfigurations` output, for example, these would be all the NixOS options—and then the [Nix CLI][cli] can enable users to override options from the command line. ## Conclusion Flake schemas solve a long-standing problem with flakes: the fact that Nix currently has built-in support for a only small set of output types, which made those outputs more equal than others. With schemas, all flake output types are on an equal footing. [cli]: https://zero-to-nix.com/concepts/nix#cli [derivation]: https://zero-to-nix.com/concepts/derivations [flakehub]: https://flakehub.com [flakes]: https://zero-to-nix.com/concepts/flakes [hydra]: https://github.com/NixOS/hydra [modules]: https://zero-to-nix.com/concepts/nixos#modules [nix]: https://nixos.org [nix-develop]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-develop.html [nix-flake-check]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-check.html [nix-flake-show]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-show.html [nix-repo]: https://github.com/NixOS/nix [nixos]: https://zero-to-nix.com/concepts/nixos [nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs [output]: https://zero-to-nix.com/concepts/flakes#outputs [packages]: https://zero-to-nix.com/concepts/packages [pr]: https://github.com/NixOS/nix/pull/8892 [repo]: https://github.com/DeterminateSystems/flake-schemas --- **Introducing FlakeHub** Published: August 22, 2023 URL: https://determinate.systems/blog/introducing-flakehub Today, we at [Determinate Systems][detsys] are extremely excited to announce the release of [**FlakeHub**][flakehub], a platform for discovering and publishing [Nix flakes][flakes]. FlakeHub provides the Nix ecosystem with a variety of new [capabilities](#features): - The ability to [explore](#explore) the Nix flake landscape. - [Semantic versioning](#semver) for flakes, including version modifiers like `~` (flexible patch) and `=` (exact match). - Automated [flake publishing](#publish) with GitHub Actions. We think that FlakeHub could be a transformative force in the Nix ecosystem and provide a crucial inflection point for flake adoption within and outside of the Nix community. We can't wait to see what people do with it. Check out our flake publishing wizard at [flakehub.com/new][wizard] if you want to get started now or read on to learn more. ## What FlakeHub offers \{#features} ### A world of flakes \{#explore} FlakeHub enables you to explore the current universe of Nix flakes in a variety of new ways. **Search**. With FlakeHub, you can search all published flakes by publisher, project, description, and tags. To activate the FlakeHub search widget you can either click on the magnifying glass icon in the navbar or press Cmd + K on macOS or Ctrl + K on Linux, or **List all flakes**. Go to [flakehub.com/flakes][all-flakes] to see a listing of all published flakes. **Organizations**. Go to [flakeshub.com/orgs][orgs] to see all the organizations that have published flakes. **Tags**. You can use the `/tag/:tag` endpoint to search for flakes by tag. [flakehub.com/tag/nixos][nixos-tag], for example, shows you all flakes with the `nixos` tag. ### Semantic versioning \{#semver} Flakes are currently rooted in [revisions]. Revision hashes are valuable because they're highly granular; _any_ change in the contents of the objects in the commit produces a new revision ID. But Nix, even with flakes, doesn't have a built-in concept of _versions_. As a refresher, here's what semantic versions look like: SemVer is popular because it's expressive. Major, minor, and patch are straightforward markers (hence the "semantic"). The difference between version 1.4.1 and 1.4.2 varies based on the project, and it isn't always clear what should constitute a patch version versus a minor version, but it at least provides _a_ framework for making those judgments. Revision hashes do not. In the Nix ecosystem, upgrading a [flake input][input] using [`nix flake update`][nix-flake-update] has thus far meant switching to the most recent Git revision for that [reference][references]. FlakeHub changes that by embedding semantic versioning directly into [flake references][references]. Here's the basic structure: Here are some example [`nix flake metadata`][nix-flake-metadata] commands that illustrate what FlakeHub makes possible: ```shell # See the most recent Nixpkgs stable nix flake metadata "https://flakehub.com/f/NixOS/nixpkgs/*.tar.gz" # See the most recent Nixpkgs under major version 0.2305 nix flake metadata "https://api.flakehub.com/f/NixOS/nixpkgs/0.2305.tar.gz" # See the most recent Nixpkgs from NixOS 0.2305, the current stable release nix flake metadata "https://flakehub.com/f/NixOS/nixpkgs/0.2305.x.tar.gz" # See the most recent Nixpkgs under minor version 0.2305 nix flake metadata "https://api.flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz" ``` In addition to `*`, FlakeHub also supports the `~` and `=` operators. This URL denotes the most recent minor version using `~`: To see the metadata: ```shell nix flake metadata "https://flakehub.com/f/NixOS/nixpkgs/~0.2305.tar.gz" ``` This URL denotes an exact version using `=`: To see the metadata: ```shell nix flake metadata "https://flakehub.com/f/NixOS/nixpkgs/=0.2305.tar.gz" ``` Now let's see how this looks in a `flake.nix` file: ```nix { inputs.nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz"; outputs = { self, nixpkgs }: { # Use the nixpkgs input here }; } ``` Now when you run [`nix flake show`][nix-flake-show] in this directory, the `flake.lock` will pin [Nixpkgs] to the most recent revision under 0.2305. When you run [`nix flake update`][nix-flake-update], it will stay in that series. ### Publish your flakes \{#publish} You can publish flakes to FlakeHub using [GitHub Actions][actions]. To automatically publish your flake every time you push a new tag, you can add this [workflow config][workflow] to your project: ```yaml name: Push flake to FlakeHub on: push: tags: - "v*.*.*" jobs: flakehub: runs-on: ubuntu-22.04 permissions: id-token: write contents: read steps: - uses: DeterminateSystems/determinate-nix-action@v3 - uses: actions/checkout@v3 - name: Push to FlakeHub uses: determinatesystems/flakehub-push@main with: visibility: "public" ``` To adopt a "rolling" strategy: ```yaml on: push: branches: - main jobs: flakehub: # Other configs from above - name: Push to FlakeHub uses: determinatesystems/flakehub-push@main with: rolling: true visibility: "public" ``` This sets a rolling prefix to be suffixed with the commit count. `v0.1`, for example, would be suffixed with `.123`, producing a release named `v0.1.123-{revision}`. We've also provided a handy publishing wizard at [flakehub.com/new][wizard] that walks you through the process step by step. ## You should adopt flakes Here at Determinate Systems, we are 100% all in on [Nix flakes][flakes] because we firmly believe that they are the future of Nix. Here are some reasons why: - Flakes overcome _all_ of the deficiencies of [Nix channels][channels], most importantly surrounding [pinning] to specific revisions of your code via the [`flake.lock`][lock] file. - Flakes make Nix expressions dramatically more introspectable through commands like [`nix flake show`][nix-flake-show] and [`nix flake metadata`][nix-flake-metadata]. - Flake [references] provide a convenient universal mechanism for specifying the location of a flake. - Flakes provide standard way to structure Nix [outputs]. - In place of a mélange of files like `default.nix`, `configuration.nix`, and `shell.nix`, flakes are standardized a single `flake.nix` file as the entry point of a project. And our confidence in flakes isn't just rhetorical. We've acted on it: - [The Determinate Nix Installer][nix-installer], our unofficial, still-experimental installer for Nix, enables flakes by default. - [Zero to Nix][z2n], an opinionated learning resource for Nix that we created, is centered around flakes and actively promotes beginning your Nix journey with flakes rather than [channels]. - We created both [Flake Checker][checker], a tool that performs "health checks" on your [`flake.lock`][lock] files, and the [`update-flake-lock`][update] Github Action, which automates `flake.lock` updates for GitHub projects. FlakeHub is our most ambitious salvo yet and part of a [broader story][better] that guides everything we do as a company. ## Current Nix practices SemVer has become standard practice in most programming language communities: JavaScript, Python, Rust, Java, Ruby, and on down the line. Noticeably absent from this list: **Nix**. Prior to the introduction of [flakes], most Nix development used [Nixpkgs] as the only external Nix input. Typically, developers would subscribe to a [channel][channels] of Nixpkgs, like `22.11` or `23.05`, continuously update to the most recent revision in that channel, and then switch channels every six months. The introduction of flakes began to change things. Because flakes provide primitives to compose and [pin][pinning] your Nix inputs, we began to see a wider variety of inputs in Nix projects—a decentralization of the Nix universe. But even with flakes, Nix development still typically involves one of two approaches to handling Nix flake inputs: - Give me the most recent revision in this specific branch. And there are only a few prominent flakes, like [Nixpkgs], where branches, such as `nixpkgs-unstable`, are used. More often the approach is... - Just give me the most recent revision. The consequence is that operations like [`nix flake update`][nix-flake-update] are too often fraught with peril. You run it, you hope for the best, you try to fix what breaks, and you steel yourself to do the same over and over again in the future. We hope that FlakeHub and the introduction of [SemVer](#semver) to [flakes] enables the Nix community to move beyond this state of affairs and into harmony with other languages. We want `nix flake update` to feel a lot more like housekeeping and a lot less like a venturesome leap into the unknown. ## How it works \{#how} In order to make semantic versions work in FlakeHub, we submitted a change to Nix that enables it to handle arbitrary URL endpoints. If the endpoint returns an [HTTP 301][301] status code and a `Link` header, Nix fetches the flake tarball from that URL and records that URL in the `flake.lock` file. See [the FlakeHub docs][upgrade-nix] for information on which versions of Nix support this. Take this `flake.nix` file: ```nix { inputs.nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.490090.tar.gz"; # Flake outputs here } ``` If you run a `nix flake` command like [`nix flake show`][nix-flake-show] in this directory, the generated `flake.lock` looks like this: ```json { "nodes": { "nixpkgs": { "locked": { "narHash": "sha256-7el+r373PubFExJSr/FA4wwr66ekBn4afDJuEiuefO8=", "rev": "475d5ae2c4cb87b904545bdb547af05681198fcc", "revCount": 490311, "type": "tarball", "url": "https://api.flakehub.com/f/pinned/NixOS/nixpkgs/0.2305.490311%2Brev-475d5ae2c4cb87b904545bdb547af05681198fcc/018a1928-9148-72c2-b387-ee7e2286c1b9/source.tar.gz" }, "original": { "type": "tarball", "url": "https://api.flakehub.com/f/NixOS/nixpkgs/0.2305.%2A.tar.gz" } }, "root": { "inputs": { "nixpkgs": "nixpkgs" } } }, "root": "root", "version": 7 } ``` As you can see, Nix has resolved the URL into a specific tarball reference using the FlakeHub server's HTTP response. FlakeHub's implementation includes not just Git revision information but also a UUID in the tarball URL. And FlakeHub is just one implementation. With this feature now in Nix, you can build your own HTTP servers to serve flakes according to whatever scheme you wish, SemVer or not. ## Room for improvement There are a few areas where FlakeHub isn't quite where we want it to be yet: - **CLI tool**. In the coming weeks, we intend to introduce a CLI tool that enables you to perform a variety of actions against the FlakeHub API, such as search. - **Private flakes**. Upon launch, all the flakes on FlakeHub are either [public or unlisted][visibility]; unlisted flakes can be used by Nix but don't show up on the website. Down the road, we want to enable you to make flakes available only to people within your org. ## Implications We're confident that FlakeHub will be a major boon to flakes adoption and to the Nix ecosystem more broadly. A dedicated home for flakes will enable people to see which flakes exist and add them to their daily workflows more easily than ever before—and within those workflows, the update path will be predictable in a way that we haven't yet seen in the world of Nix. Whether SemVer takes hold in Nix is up to you. But we're confident that adoption will happen swiftly now that Nix users have the means to adopt it. Take a look at [**flakehub.com**][flakehub], browse the docs at [docs.determinate.systems][docs], and [publish your flake][wizard] today. Or join us [on Discord][discord] if you have questions or problems or just want to connect with other Nix users. [301]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/301 [actions]: https://github.com/features/actions [all-flakes]: https://flakehub.com/flakes [better]: /blog/we-want-to-make-nix-better [channels]: https://zero-to-nix.com/concepts/channels [checker]: https://github.com/DeterminateSystems/flake-checker [detsys]: https://determinate.systems [discord]: https://discord.gg/invite/a4EcQQ8STr [docs]: https://docs.determinate.systems [flakehub]: https://flakehub.com [flakes]: https://zero-to-nix.com/concepts/flakes [input]: https://zero-to-nix.com/concepts/flakes#inputs [nix-installer]: https://github.com/determinateSystems/nix-installer [lock]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-lock.html [nix-flake-metadata]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-metadata.html [nix-flake-show]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-show.html [nix-flake-update]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-update.html [nixos-tag]: https://flakehub.com/tag/nixos [nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs [orgs]: https://flakehub.com/orgs [outputs]: https://zero-to-nix.com/concepts/flakes#outputs [pinning]: https://zero-to-nix.com/concepts/pinning [references]: https://zero-to-nix.com/concepts/flakes#references [revisions]: https://zero-to-nix.com/concepts/flakes#revisions [update]: https://github.com/determinateSystems/update-flake-lock [upgrade-nix]: https://docs.determinate.systems/determinate-nix#determinate-nixd-upgrade [visibility]: https://docs.determinate.systems/flakehub/concepts/visibility [wizard]: https://flakehub.com/new [workflow]: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions [z2n]: https://zero-to-nix.com --- **Instrumenting Axum projects** Published: July 20, 2023 URL: https://determinate.systems/blog/instrumenting-axum Recently I was helping someone get bootstrapped on their Axum project, and they were getting a bit frustrated with the lack of context about the goings-on in their application. While Axum and its ecosystem are well instrumented, by default most of it is not well surfaced for beginners. It can be a bit intimidating figuring out how to put the pieces together and use them. Once we set up [`tracing`] and [`color_eyre`] our logging and errors went from looking like this: ``` Listening on [::]:8080 Got an error request on `/error` Got request for `/favicon.ico`, but no route was found. Got a homepage request on `/` ``` To something with a little bit more context: ``` INFO Listening on [::]:8080 INFO request:get_error: Got an error request uri=/error method=GET source=::1 ERROR request: 0: Whoopsies ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0: demo::routes::get_error at src/routes.rs:11 1: demo::trace_layer::request with uri=/error method=GET source=::1 at src/trace_layer.rs:24 Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Run with RUST_BACKTRACE=full to include source snippets. uri=/error method=GET source=::1 INFO request:get_home: Got a homepage request uri=/ method=GET source=::1 INFO request:fallback_404: Got request, but no route was found. uri=/favicon.ico method=GET source=::1 ``` While this isn't a complete and fully mature solution suitable for an industrial application, it is a good starting point for them that smoothly evolve into something more mature. Often the first place I start with a binary project is the command line interface, so we'll start there. ## Making instrumentation tuneable Instrumentation can have many knobs, this project used [`clap`], so we took the existing `Cli` struct and added an `instrumentation` field: ```rust // src/cli/mod.rs mod instrumentation; mod logger; use clap::Parser; use std::net::{IpAddr, Ipv6Addr, SocketAddr}; #[derive(Parser)] pub(crate) struct Cli { #[clap(long, env = "DEMO_BIND", default_value_t = SocketAddr::new( IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0, 0, 0)), 8080) )] pub(crate) bind: SocketAddr, #[clap(flatten)] pub(crate) instrumentation: instrumentation::Instrumentation, } ``` Using `#[clap(flatten)]` here means we can define all our instrumentation options in a separate struct, while still appearing in the `--help` as one might expect: ``` ❯ cargo run -- --help Finished dev [unoptimized + debuginfo] target(s) in 0.04s Running `target/debug/demo --help` Usage: demo [OPTIONS] Options: --bind [env: DEMO_BIND=] [default: [::]:8080] -v, --verbose... Enable debug logs, -vv for trace [env: DEMO_VERBOSITY=] --logger Which logger to use [env: DEMO_LOGGER=] [default: compact] [possible values: compact, full, pretty, json] --log-directive [...] Tracing directives See https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#directives [env: DEMO_LOG_DIRECTIVES=] -h, --help Print help (see a summary with '-h') ``` Tracing offers use the choice of several built in logging styles, or even our own logging style if we so choose! It's straightforward to pass on this choice to the user, though we do need to create `Logger` enum to represent that choice: ```rust // src/cli/logger.rs #[derive(Clone, Default, Debug, clap::ValueEnum)] pub(crate) enum Logger { #[default] Compact, Full, Pretty, Json, } impl std::fmt::Display for Logger { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { let logger = match self { Logger::Compact => "compact", Logger::Full => "full", Logger::Pretty => "pretty", Logger::Json => "json", }; write!(f, "{}", logger) } } ``` The `Logger` can then get used in our `Instrumentation` struct, which we'll build up over the next few code blocks: ```rust // src/cli/instrumentation.rs use color_eyre::eyre::WrapErr; use std::{error::Error, io::IsTerminal}; use tracing::Subscriber; use tracing_subscriber::{ filter::Directive, layer::{Layer, SubscriberExt}, registry::LookupSpan, util::SubscriberInitExt, EnvFilter, }; use super::logger::Logger; #[derive(clap::Args, Debug, Default)] pub(crate) struct Instrumentation { /// Enable debug logs, -vv for trace #[clap( short = 'v', env = "DEMO_VERBOSITY", long, action = clap::ArgAction::Count, global = true )] pub verbose: u8, /// Which logger to use #[clap( long, env = "DEMO_LOGGER", default_value_t = Default::default(), global = true )] pub(crate) logger: Logger, /// Tracing directives /// /// See https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#directives #[clap(long = "log-directive", global = true, env = "DEMO_LOG_DIRECTIVES", value_delimiter = ',', num_args = 0..)] pub(crate) log_directives: Vec, } impl Instrumentation { pub(crate) fn log_level(&self) -> String { match self.verbose { 0 => "info", 1 => "debug", _ => "trace", } .to_string() } // (continued below) } ``` This interface offers users the ability to specify 'traditional' verbosity options such as `-v` or, my favorite, `-vvvvvvv` through the use of [`clap::ArgAction::Count`]. It also permits the use of more [`tracing`]-specific options like the `Logger` we created above, or the option of any number of [`tracing_subscriber::filter::Directive`]. We'll explore those a bit together once we get things running. We can then attach some functions to this struct that set up a [`Registry`][`tracing_subscriber::registry::Registry`] which stores [span][`tracing::span`] data, such as the data defined in [`#[tracing::instrument]`][`tracing::instrument`] calls. In a `setup()` function we'll build a [`Registry`][`tracing_subscriber::registry::Registry`] and compose it with several layers, including a [`ErrorLayer`][`tracing_error::ErrorLayer`] and an [`EnvFilter`][`tracing_subscriber::filter::EnvFilter`] layer we configure from the knobs made available in the `Cli` struct (as well as some conventional environment variables): ```rust impl Instrumentation { // (continued) pub(crate) fn setup(&self) -> color_eyre::Result<()> { let filter_layer = self.filter_layer()?; let registry = tracing_subscriber::registry() .with(filter_layer) .with(tracing_error::ErrorLayer::default()); // `try_init` called inside `match` since `with` changes the type match self.logger { Logger::Compact => { registry.with(self.fmt_layer_compact()).try_init()? } Logger::Full => { registry.with(self.fmt_layer_full()).try_init()? } Logger::Pretty => { registry.with(self.fmt_layer_pretty()).try_init()? } Logger::Json => { registry.with(self.fmt_layer_json()).try_init()? } } Ok(()) } pub(crate) fn filter_layer(&self) -> color_eyre::Result { let mut filter_layer = match EnvFilter::try_from_default_env() { Ok(layer) => layer, Err(e) => { // Catch a parse error and report it, ignore a missing env if let Some(source) = e.source() { match source.downcast_ref::() { Some(std::env::VarError::NotPresent) => (), _ => return Err(e).wrap_err_with(|| "parsing RUST_LOG directives"), } } // If the `--log-directive` is specified, don't set a default if self.log_directives.is_empty() { EnvFilter::try_new(&format!( "{}={}", env!("CARGO_PKG_NAME").replace('-', "_"), self.log_level() ))? } else { EnvFilter::try_new("")? } } }; for directive in &self.log_directives { let directive_clone = directive.clone(); filter_layer = filter_layer.add_directive(directive_clone); } Ok(filter_layer) } // (continued below) } ``` Then we can go ahead and define the various format layers, along with any [customization](https://docs.rs/tracing-subscriber/latest/tracing_subscriber/fmt/struct.Layer.html#) we wanted to do. ```rust impl Instrumentation { // (continued) pub(crate) fn fmt_layer_full(&self) -> impl Layer where S: Subscriber + for<'span> LookupSpan<'span>, { tracing_subscriber::fmt::Layer::new() .with_ansi(std::io::stderr().is_terminal()) .with_writer(std::io::stderr) } pub(crate) fn fmt_layer_pretty(&self) -> impl Layer where S: Subscriber + for<'span> LookupSpan<'span>, { tracing_subscriber::fmt::Layer::new() .with_ansi(std::io::stderr().is_terminal()) .with_writer(std::io::stderr) .pretty() } pub(crate) fn fmt_layer_json(&self) -> impl Layer where S: Subscriber + for<'span> LookupSpan<'span>, { tracing_subscriber::fmt::Layer::new() .with_ansi(std::io::stderr().is_terminal()) .with_writer(std::io::stderr) .json() } pub(crate) fn fmt_layer_compact(&self) -> impl Layer where S: Subscriber + for<'span> LookupSpan<'span>, { tracing_subscriber::fmt::Layer::new() .with_ansi(std::io::stderr().is_terminal()) .with_writer(std::io::stderr) .compact() .without_time() .with_target(false) .with_thread_ids(false) .with_thread_names(false) .with_file(false) .with_line_number(false) } } ``` Later, your project may choose to integrate something like [`tracing_opentelemetry`] and route that data to a service like [Honeycomb]. ## Building up `Error`s [`axum`] allows us to define routes that can return errors, if that error implements [`axum::response::IntoResponse`]: ```rust async fn some_route() -> Result { Ok("I work ok!") } ``` Unfortunately, this does not work out of the box with [`eyre::Report`]. It's also forbidden for us (as `demo`) to implement [`IntoResponse`][`axum::response::IntoResponse`] for [`Report`][`eyre::Report`] due to the [Orphan Rule](https://rust-lang.github.io/chalk/book/clauses/coherence.html), so we must create a wrapper. While we do that, we might as well create a crate-specific `DemoError` as well, which can be later used to define user-facing error messages and status codes. ```rust // src/error.rs use axum::{ http::StatusCode, response::{IntoResponse, Response}, }; pub type Result = color_eyre::Result; // A generic error report // Produced via `Err(some_err).wrap_err("Some context")` // or `Err(color_eyre::eyre::Report::new(SomeError))` pub struct Report(color_eyre::Report); impl std::fmt::Debug for Report { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { self.0.fmt(f) } } impl From for Report where E: Into, { fn from(err: E) -> Self { Self(err.into()) } } // Tell axum how to convert `Report` into a response. impl IntoResponse for Report { fn into_response(self) -> Response { let err = self.0; let err_string = format!("{err:?}"); tracing::error!("{err_string}"); if let Some(err) = err.downcast_ref::() { return err.response() } // Fallback ( StatusCode::INTERNAL_SERVER_ERROR, "Something went wrong".to_string(), ) .into_response() } } #[derive(thiserror::Error, Debug)] pub(crate) enum DemoError { #[error("A spooky thing happened")] Spooky, } // Tell axum how to convert `DemoError` into a response. impl DemoError { fn response(&self) -> Response { match self { Self::Spooky => ( StatusCode::IM_A_TEAPOT, "A user-facing message about a Spooky".to_string(), ) .into_response(), } } } ``` Now we can use that new `Result` in our routes, and return errors either ad-hoc via [`eyre::eyre`], or structured through the error type we created with [`thiserror`]. > This is also a good opportunity to decorate your routes with a [`tracing::instrument`] attribute. > > Consider using `skip_all` and explicitly defining any fields if you are dealing complex route arguments! > Eg. `#[tracing::instrument(skip_all, fields(uid = context.user.uid))]`. ```rust use crate::error::Result; use axum::response::IntoResponse; use color_eyre::eyre::eyre; #[tracing::instrument] pub(crate) async fn get_error() -> Result { tracing::info!("Got an error request"); // Bang!!! Err(eyre!("Whoopsies"))?; Ok(()) } ``` At the start, and during prototyping, you can use [`eyre::eyre`] to write ad-hoc errors. As the application grows, you'll be able to work with [`thiserror`] and use [`downcast_ref`](https://docs.rs/eyre/latest/eyre/struct.Report.html#method.downcast_ref) (like we did in the `instrumentation.rs` code) and other tools available in [`eyre::Report`] to respond with a more structured and informative messages to the web browser. ```rust use crate::error::Result; use axum::response::IntoResponse; // After adding a `Spooky` variant to `DemoError` in `src/error.rs`: // // #[derive(thiserror::Error, Debug)] // enum DemoError { // #[error("A spooky thing happened")] // Spooky // } #[tracing::instrument] pub(crate) async fn get_demo_error() -> Result { tracing::info!("Got an error request"); // Bang!!! Err(DemoError::Spooky)?; Ok(()) } ``` ## Putting the pieces together After painting our user interface and performing the ritualistic type dancing, it's finally time to update `main()` with the code to get everything working. At the top of `main()` we want to install [`color_eyre`]'s error hooks, so that any errors after that are fully styled and integrated. After that, we can call `setup()` on the `Instrumentation` instance created by [`clap`] to initialize tracing. In order to add the appropriate spans to requests, [`tower_http::trace::TraceLayer`] should be added to the [`axum::routing::Router`]: ```rust // src/main.rs mod cli; mod error; mod routes; mod trace_layer; use crate::{cli::Cli, error::Result}; use axum::routing::get; use clap::Parser; use std::{io::IsTerminal, net::SocketAddr, process::ExitCode}; use tower_http::trace::TraceLayer; #[tokio::main] async fn main() -> Result { color_eyre::config::HookBuilder::default() .theme(if !std::io::stderr().is_terminal() { // Don't attempt color color_eyre::config::Theme::new() } else { color_eyre::config::Theme::dark() }) .install()?; let cli = Cli::parse(); cli.instrumentation.setup()?; let trace_layer = TraceLayer::new_for_http() .make_span_with(trace_layer::trace_layer_make_span_with) .on_request(trace_layer::trace_layer_on_request) .on_response(trace_layer::trace_layer_on_response); let app = axum::Router::new() .route("/", get(routes::get_home)) .route("/errors", get(routes::get_error)) .route("/errors/demo", get(routes::get_demo_error)) .fallback(routes::fallback_404) .layer(trace_layer); tracing::info!("Listening on {}", cli.bind); axum::Server::bind(&cli.bind) .serve(app.into_make_service_with_connect_info::()) .await?; Ok(ExitCode::SUCCESS) } ``` [`TraceLayer`][`tower_http::trace::TraceLayer`] offers opportunities to attach our own functions where we can create spans or emit events. There exist defaults ([`tower_http::trace::{DefaultMakeSpan, DefaultOnRequest}`][`tower_http::trace`]) which can be modified using a builder API, these have the rather unfortunate quality of being part of the `tower_http` crate, not our own, so because the defaults we configured in the `Instrumentation::filter_layer()` function they won't normally be enabled. Defining our own functions allows us to build our own base span, as well as ensure these spans are part of our own crate. If your project opts to use the defaults, consider altering `Instrumentation::filter_layer()` to also set some default for `tower_http`. ```rust // src/trace_layer.rs use axum::{body::BoxBody, extract::ConnectInfo, response::Response}; use hyper::{Body, Request}; use std::{net::SocketAddr, time::Duration}; use tracing::Span; pub(crate) fn trace_layer_make_span_with(request: &Request) -> Span { tracing::error_span!("request", uri = %request.uri(), method = %request.method(), // This is not particularly robust, but suitable for a demo // You'll need to change this if you deploy behind a proxy // (eg the `X-forwarded-for` header) source = request.extensions() .get::>() .map(|connect_info| tracing::field::display(connect_info.ip().to_string()), ).unwrap_or_else(|| tracing::field::display(String::from("")) ), // Fields must be defined to be used, define them as empty if they populate later status = tracing::field::Empty, latency = tracing::field::Empty, ) } pub(crate) fn trace_layer_on_request(_request: &Request, _span: &Span) { tracing::trace!("Got request") } pub(crate) fn trace_layer_on_response( response: &Response, latency: Duration, span: &Span, ) { span.record( "latency", tracing::field::display(format!("{}μs", latency.as_micros())), ); span.record("status", tracing::field::display(response.status())); tracing::trace!("Responded"); } ``` We can create a few test routes a fallback route, one reporting no error, another reporting an [`eyre::eyre`] based error, and a route reporting a `DemoError`. Due to our error handling code we can use `DemoError` variants to return specific, user-facing messages and status codes, while more ad-hoc or developer/operator facing errors can be [`eyre::eyre`] based. ```rust // src/routes.rs use crate::error::{Result, DemoError}; use axum::response::IntoResponse; use color_eyre::eyre::eyre; #[tracing::instrument] pub(crate) async fn get_home() -> Result { tracing::info!("Got a homepage request"); Ok("Welcome to my super cute home page") } #[tracing::instrument] pub(crate) async fn get_error() -> Result { tracing::info!("Got an error request"); Err(eyre!("Whoopsies"))?; Ok(()) } #[tracing::instrument] pub(crate) async fn get_demo_error() -> Result { tracing::info!("Got an error request"); Err(DemoError::Spooky)?; Ok(()) } #[tracing::instrument(skip_all)] pub(crate) async fn fallback_404() -> Result { tracing::info!("Got request, but no route was found."); Ok("You failed to find my super cute home page") } ``` If you're following along you may have run `cargo add` for some of the dependencies above, the specific examples shown utilized these features and crates: ```toml # Cargo.toml [package] name = "demo" version = "0.1.0" edition = "2021" [dependencies] axum = "0.6" clap = { version = "4.3", features = ["derive", "env"] } color-eyre = { version = "0.6", default-features = false, features = [ "issue-url", "tracing-error", "capture-spantrace", "color-spantrace" ] } hyper = "0.14" thiserror = "1" tokio = { version = "1", features = ["full"] } tower = "0.4" tower-http = { version = "0.4", features = ["trace"] } tracing = "0.1" tracing-error = "0.2" tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] } ``` One feature you may wish to enable the `track-caller` on [`color-eyre`][`color_eyre`] package, it will show the locations of errors in the output using the [`track_caller`] attribute. The reported location is not always accurate, but often it's helpful. ## Going for a test run At this point, we can run our binary and visit the homepage: ```bash ❯ cargo run Finished dev [unoptimized + debuginfo] target(s) in 0.05s Running `target/debug/demo` INFO Listening on [::]:8080 INFO request:get_home: Got a homepage request uri=/ method=GET source=::1 INFO request:fallback_404: Got request, but no route was found. uri=/favicon.ico method=GET source=::1 ``` We can run the application again with different directives and get some more info from one of the packages, for example [`hyper`] which does the bulk of the HTTP work: ```bash ❯ cargo run -- --logger compact --log-directive demo=trace --log-directive hyper=trace Finished dev [unoptimized + debuginfo] target(s) in 0.05s Running `target/debug/demo --logger compact --log-directive demo=trace --log-directive hyper=trace` INFO Listening on [::]:8080 TRACE Conn::read_head TRACE received 454 bytes TRACE parse_headers: Request.parse bytes=454 TRACE parse_headers: Request.parse Complete(454) DEBUG parsed 12 headers DEBUG incoming body is empty TRACE request: Got request uri=/ method=GET source=::1 INFO request:get_home: Got a homepage request uri=/ method=GET source=::1 TRACE request: Responded uri=/ method=GET source=::1 latency=175μs status=200 OK TRACE encode_headers: Server::encode status=200, body=Some(Known(34)), req_method=Some(GET) TRACE sized write, len = 34 TRACE buffer.queue self.len=117 buf.len=34 # ... ``` That's **a lot of output**! We can also use [log directives][`tracing_subscriber::filter::EnvFilter`] to drill down into specific parts of the code, this can be quite useful for debugging. Here we isolate [`hyper`] events to `parse_headers`: ```bash ❯ cargo run -- --log-directive hyper[parse_headers]=trace --log-directive demo=info Finished dev [unoptimized + debuginfo] target(s) in 0.05s Running `target/debug/demo --log-directive 'hyper[parse_headers]=trace' --log-directive demo=info` INFO Listening on [::]:8080 TRACE parse_headers: Request.parse bytes=464 TRACE parse_headers: Request.parse Complete(464) INFO request:get_demo_error: Got an error request uri=/error/demo method=GET source=::1 ERROR request: 0: A spooky thing happened ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0: demo::routes::get_demo_error at src/routes.rs:20 1: demo::trace_layer::request with uri=/error/demo method=GET source=::1 at src/trace_layer.rs:24 Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Run with RUST_BACKTRACE=full to include source snippets. uri=/error/demo method=GET source=::1 ``` Or how about only logging requests for a specific URL? We can specify a filter like `demo[request{uri=/}]=trace` and visit several pages, observing we only see logs for the one we filtered for: ```bash ❯ cargo run -- --log-directive demo[request{uri=/}]=trace Compiling demo v0.1.0 (/home/ana/git/determinatesystems/axum-with-tracing-and-eyre) Finished dev [unoptimized + debuginfo] target(s) in 2.25s Running `target/debug/demo --log-directive 'demo[request{uri=/}]=trace'` TRACE request: Got request uri=/ method=GET source=::1 INFO request:get_home: Got a homepage request uri=/ method=GET source=::1 TRACE request: Responded uri=/ method=GET source=::1 latency=333μs status=200 OK ``` Of the different logger options, my personal favorite is the [`pretty`][`tracing_subscriber::fmt::format::Pretty`] logger which creates a cozy, human readable experience: ```bash ❯ cargo run -- --logger pretty Finished dev [unoptimized + debuginfo] target(s) in 0.04s Running `target/debug/demo --logger pretty` 2023-07-19T21:56:16.481962Z INFO demo: Listening on [::]:8080 at src/main.rs:38 2023-07-19T21:56:20.084723Z INFO demo::routes: Got a homepage request at src/routes.rs:7 in demo::routes::get_home in demo::trace_layer::request with uri: /, method: GET, source: ::1 2023-07-19T21:56:20.131037Z INFO demo::routes: Got request, but no route was found. at src/routes.rs:40 in demo::routes::fallback_404 in demo::trace_layer::request with uri: /favicon.ico, method: GET, source: ::1 ``` The tracing spans also appear in our errors, let's take a look at those. We can observe these errors when we visit the respective `/error` urls: ```bash ❯ cargo run Finished dev [unoptimized + debuginfo] target(s) in 0.04s Running `target/debug/demo` INFO Listening on [::]:8080 INFO request:get_error: Got an error request uri=/error method=GET source=::1 ERROR request: 0: Whoopsies ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0: demo::routes::get_error at src/routes.rs:12 1: demo::trace_layer::request with uri=/error method=GET source=::1 at src/trace_layer.rs:24 Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Run with RUST_BACKTRACE=full to include source snippets. uri=/error method=GET source=::1 INFO request:fallback_404: Got request, but no route was found. uri=/favicon.ico method=GET source=::1 INFO request:get_demo_error: Got an error request uri=/error/demo method=GET source=::1 ERROR request: 0: A spooky thing happened ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0: demo::routes::get_demo_error at src/routes.rs:27 1: demo::trace_layer::request with uri=/error/demo method=GET source=::1 at src/trace_layer.rs:24 Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Run with RUST_BACKTRACE=full to include source snippets. uri=/error/demo method=GET source=::1 INFO request:fallback_404: Got request, but no route was found. uri=/favicon.ico method=GET source=::1 ``` Visiting `/error/demo` we can observe the browser returning `A spooky occured` as we intended. ## Conclusion The combination of [`thiserror`], [`tracing`], and [`color_eyre`] provides a solid starting point for a budding project. The ability to instrument code with spans, see those spans in errors, then filter based on those spans in logging messages enables greater insight while diagnosing issues and authoring new features. As your project grows, these same tools offer a smooth path to adoption of standards like [OpenTelemetry]. While researching this article I bumped into the really lovely article by [@carlosmv](https://carlosmv.hashnode.dev/adding-logging-and-tracing-to-an-axum-app-rust) about a many of the same ideas! They go in detail about how to use [`tracing_appender`] and some other things which are not covered here, but don't discuss error handling as much. If you want to dig deeper into this topic, that article is a wonderful place to visit after this! A few years ago I gave [a recorded talk at TremorCon](https://www.youtube.com/watch?v=ZC7fyqshun8) which discussed [`tracing`] and [`color_eyre`], and this article expands on several of the concepts mentioned there. [`axum::response::IntoResponse`]: https://docs.rs/axum/0.6/axum/response/trait.IntoResponse.html [`axum::routing::Router`]: https://docs.rs/axum/0.6/axum/routing/struct.Router.html [`axum`]: https://docs.rs/axum/0.6/axum/ [`clap::ArgAction::Count`]: https://docs.rs/clap/4.3/clap/enum.ArgAction.html#variant.Count [`clap`]: https://docs.rs/clap/4.3/clap/ [`color_eyre`]: https://docs.rs/color-eyre/0.6/color_eyre/ [`eyre::eyre`]: https://docs.rs/eyre/0.6/eyre/macro.eyre.html [`eyre::Report`]: https://docs.rs/eyre/0.6/eyre/struct.Report.html [`hyper`]: https://docs.rs/hyper/0.14/hyper/ [`thiserror`]: https://docs.rs/thiserror/1/thiserror/ [`tower_http::trace::TraceLayer`]: https://docs.rs/tower-http/0.4.1/tower_http/trace/struct.TraceLayer.html [`tower_http::trace`]: https://docs.rs/tower-http/0.4/tower_http/trace/index.html [`tracing_appender`]: https://docs.rs/tracing-appender/latest/tracing_appender/ [`tracing_error::ErrorLayer`]: https://docs.rs/tracing-error/0.2/tracing_error/struct.ErrorLayer.html [`tracing_opentelemetry`]: https://docs.rs/tracing-opentelemetry/latest/tracing_opentelemetry/ [`tracing_subscriber::filter::Directive`]: https://docs.rs/tracing-subscriber/0.3/tracing_subscriber/filter/struct.Directive.html [`tracing_subscriber::filter::EnvFilter`]: https://docs.rs/tracing-subscriber/0.3/tracing_subscriber/filter/struct.EnvFilter.html [`tracing_subscriber::fmt::format::Pretty`]: https://docs.rs/tracing-subscriber/0.3/tracing_subscriber/fmt/format/struct.Pretty.html [`tracing_subscriber::registry::Registry`]: https://docs.rs/tracing-subscriber/0.3/tracing_subscriber/registry/struct.Registry.html [`tracing::instrument`]: https://docs.rs/tracing/0.1/tracing/attr.instrument.html [`tracing::span`]: https://docs.rs/tracing/latest/tracing/span/index.html [`tracing`]: https://docs.rs/tracing/0.1/tracing/ [`track_caller`]: https://rustc-dev-guide.rust-lang.org/backend/implicit-caller-location.html [Honeycomb]: https://www.honeycomb.io/ [OpenTelemetry]: https://opentelemetry.io/ --- **Introducing the Magic Nix Cache** Published: June 26, 2023 URL: https://determinate.systems/blog/magic-nix-cache We at [Determinate Systems][detsys], along with our collaborator [Zhaofeng Li][zhaofeng], are thrilled to announce the release of the [**Magic Nix Cache**][action], a [GitHub Action][actions] that can dramatically speed up your [Nix]-related workflows. You can use it today—no signup, setup, or cost—by adding just one line to your YAML configuration: ```yaml - uses: DeterminateSystems/magic-nix-cache-action@v2 ``` Here's a more complete example: ```yaml jobs: build: steps: - uses: actions/checkout@v4 - name: Install Nix uses: DeterminateSystems/determinate-nix-action@v3 - name: Run the Magic Nix Cache uses: DeterminateSystems/magic-nix-cache-action@v2 - name: Build server and client run: | nix build .#server nix build .#client ``` We're confident that its power and ease of use will provide immediate benefits to the Nix community at large. ## Why caching is important We at [Determinate Systems][detsys] use [Nix] for all of our [GitHub Actions][actions] builds. Synchronizing environments [across dev, prod, and CI][dev] is, after all, one of Nix's major [strong points][features]. But let's be frank: Nix builds can be quite slow. Nix always [realises] a [derivation]'s entire dependency tree when it builds something, and that tree can be quite vast. Fortunately, Nix is extremely good at [caching]. Because everything stored in the [Nix store][store] has a [content-derived hash][hash] in the store path, Nix can quickly determine if something needs to be built or if it can be fetched from a known [binary cache][binary] instead. Caching can cut build times by orders of magnitude (depending on the context). To reap the benefits of caching in a CI environment like GitHub Actions, you previously needed to either: 1. Run your own binary cache backed by a storage platform like [Amazon S3][s3] or S3-compatible [Minio]. 1. Use a dedicated SaaS service like [Cachix]. The Magic Nix Cache provides a new option that doesn't require you to deploy any new infrastructure or sign up for an external service. ## How it works The Magic Nix Cache relies on a daemon called [`magic-nix-cache`][magic-nix-cache], which is written in [Rust]. This daemon acts just like a standard [binary cache server][binary] from the standpoint of Nix, but with the important difference that it writes to the [GitHub Actions Cache][actions-cache] using the [Github Actions Cache API][cache-api] instead of writing to disk, as a standard binary cache would. When you add the Magic Nix Cache to an Actions workflow, here's what happens: - When the Action is triggered, it first downloads and runs the binary for the `magic-nix-cache` daemon. The daemon then fetches a list of what's currently stored in the GitHub Actions Cache. - The rest of your workflow logic, including all of your Nix commands, proceeds. - When the workflow is finished, the `magic-nix-cache` daemon calculates a list of Nix store paths that haven't yet been written to the GitHub Actions Cache and writes them. With this approach, the Magic Nix Cache caches _everything_ written to the Nix store during your workflow run. That includes your Nix sources (including bulkier things like revisions of [Nixpkgs]), [development environments][dev-env], all the intermediate dependencies from `nix build` invocations, [Docker] images you build with Nix, and more. And it does this automatically, without requiring a workflow step of this sort: ```yaml # NOT necessary so don't do this ❌ - name: Cache Nix artifacts run: ./complex-nix-cache-script.sh ``` ### Security The Magic Nix Cache provides the same [security guarantees][security] as the [Actions Cache API][cache-api], which includes **cache isolation** across different branches and tags. Concretely, this means that the cache for a project can never be polluted by a malicious pull request because workflow runs never restore caches created for child or sibling branches. If someone submits a pull request for a branch called `uh-oh-bitcoin-miner-incoming`, for example, the Magic Nix Cache _will_ indeed cache the Nix artifacts for that workflow but nothing from that cache will ever be used by your other branches, including your default branch (such as `main`). ### What about rate limits? Rate limits are a persistent concern for GitHub Actions users, and many of us have been bitten by workflow runs that fail by inadvertently exceeding those limits. The Magic Nix Cache _is_ subject to the rate limits of the [Actions Cache API][cache-api] and large projects or builds may run up against those constraints. If that happens, you'll see log messages like this: ```log error: unable to download 'http://127.0.0.1:37515/<...>': HTTP error 418 response body: GitHub API error: API error (429 Too Many Requests): StructuredApiError { message: "Request was blocked due to exceeding usage of resource 'Count' in namespace ''." } ``` Fortunately, the Magic Nix Cache handles any rate limiting gracefully. If your workflow exceeds the limit when it's pulling Nix dependencies from your Actions cache, Nix will build those dependencies rather than fetching them from the cache. And if your workflow exceeds the limit when pushing to the cache, the Magic Nix Cache will stop caching and wait for the next workflow run to push those store paths to the cache. Rate limiting may thus increase build times on specific workflow runs but it will _not_ cause your runs to fail. ## Drawbacks Because the Magic Nix Cache uses the [Actions Cache API][cache-api], the artifacts it caches are available only to workflow runs for your project. This means that: - **Caches are shared across workflow runs but not across GitHub repos**. We suspect, however, that any potential speed benefits of cross-project caching would be marginal at best. - **Caches aren't publicly available**. If you'd like your Actions workflows to be used by members of your org or Nix users more broadly, you'll need to push your Nix artifacts to a standard [binary cache][binary]. But there's nothing preventing you from using the Magic Nix Cache to speed up future Nix workflow runs _and_ pushing to a standard binary cache outside of the Actions environment. These aren't major drawbacks given the limited scope of the Action but they are worth keeping in mind. ## Kudos and thanks The Magic Nix Cache is a collaborative project with [Zhaofeng Li][zhaofeng]. Zhaofeng is a major contributor to the Nix community perhaps best known as the author of two widely used projects: - [Attic], a self-hosted, multi-tenant, [S3]-compatible Nix [binary cache][binary] - [Colmena], a [NixOps]-inspired deployment tool for [NixOS] written in [Rust] We'd like to express our deep gratitude to Zhaofeng for his tremendous work on this project. The Magic Nix Cache is part of an already-impressive legacy and we're beyond eager to see what he contributes to Nix in the coming years. ## Implications The Magic Nix Cache provides immediate benefits to your Nix workflows in GitHub Actions with few trade-offs or downsides. And while it isn't ideal [for _all_ use cases](#drawbacks), we're confident that a wide variety of Nix projects can reduce Nix-related build times in Actions by 30-50% (and in some cases even more) with no configuration changes beyond the [single line of YAML](#one-liner) shown above. As with everything we build at [Determinate Systems][detsys], we hope that this will provide a dramatically improved experience around using Nix. Our internal usage of Nix in GitHub Actions has already been radically improved by the Magic Nix Cache, and we're confident that your experience with it will mirror our own. [action]: https://github.com/DeterminateSystems/magic-nix-cache-action [actions]: https://github.com/features/actions [actions-cache]: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows [attic]: https://github.com/zhaofengli/attic [binary]: https://zero-to-nix.com/concepts/caching/#binary-caches [cache-api]: https://docs.github.com/en/rest/actions/cache [caching]: https://zero-to-nix.com/concepts/caching [cachix]: https://cachix.org [colmena]: https://github.com/zhaofengli/colmena [derivation]: https://zero-to-nix.com/concepts/derivations [detsys]: / [dev]: /blog/nix-github-actions [dev-env]: https://zero-to-nix.com/concepts/dev-env [docker]: https://docker.com [features]: https://zero-to-nix.com/#features [hash]: https://zero-to-nix.com/concepts/nix-store#store-paths [magic-nix-cache]: https://github.com/DeterminateSystems/magic-nix-cache [minio]: https://github.com/minio/minio [nix]: https://zero-to-nix.com [nixos]: https://zero-to-nix.com/concepts/nixos [nixops]: https://github.com/NixOS/nixops [nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs [realises]: https://zero-to-nix.com/concepts/realisation [s3]: https://aws.amazon.com/s3 [rust]: https://rust-lang.org [security]: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache [store]: https://zero-to-nix.com/concepts/nix-store [zhaofeng]: https://github.com/zhaofengli --- **Introducing the Nix Flake Checker** Published: June 20, 2023 URL: https://determinate.systems/blog/flake-checker Quite possibly the best thing about the [Nix] ecosystem is that there's a small army of people hard at work improving [Nixpkgs], the largest software package repository in existence and one of the most active repos on [GitHub], every single day. Not only are they constantly adding brand new packages for stuff that you might want to use—over 80,000 packages and counting!—they're also updating existing packages, which sometimes even includes fixes for critical security vulnerabilities. But to take full advantage of this steady drumbeat of progress, it's important that you follow some best practices. To help you adopt those practices, we at [Determinate Systems][detsys] have created a tool called [Nix Flake Checker][flake-checker] and we're excited to release it to the Nix community. ## What Nix Flake Checker looks for In any [flake-enabled][flakes] Nix project, Nix Flake Checker runs three checks on the root-level [Nixpkgs] dependencies in your [flake inputs][inputs]: 1. If your Nixpkgs input uses a specific Git branch, such as `nixpkgs-unstable`, it needs to be a supported release branch. Nixpkgs' release branches stop receiving updates roughly 7 months after release and then gradually become more and more out of date—read: potentially insecure—over time. Release branches are also certain to have good [binary cache][cache] coverage, which other branches can't promise. 1. Your Nixpkgs input must have been updated in the last 30 days. Because Nixpkgs sees a steady stream of community updates, the "older" your Nixpkgs revision, the less likely you'll be to benefit from these updates. 30 days is somewhat arbitrarily chosen and potentially too lax; we may make this more strict in the future. 1. If your Nixpkgs input is a GitHub repo, it needs to have the [`NixOS`][nixos-org] org as the owner. In principle, you could have a Nixpkgs input owned by someone else, for example `inputs.nixpkgs.url = "github:EvilCo/nixpkgs"`. But we don't recommend this, first because forks and other non-upstream variants of Nixpkgs can introduce security vulnerabilities and unexpected behaviors, and second because those forks are often not kept up to date (though maybe they would be if they used Nix Flake Checker :wink:). Upstream Nixpkgs isn't bulletproof—nothing in software is!—but it has a wide range of security measures in place, most notably continuous integration testing with [Hydra](https://hydra.nixos.org), that mitigate a great deal of supply chain risk. It's important to keep in mind that Nix Flake Checker checks _all_ root-level Nixpkgs inputs. In a flake input group like this, three inputs would be checked: ```nix inputs = { nixpkgs.url = "github:NixOS/nixpkgs"; nixpkgs-unstable.url = "github:NixOS/nixpkgs/nixpkgs-unstable"; nixpkgs-very-specific-ref.url = "github:NixOS/nixpkgs/0b23874b968c333abd4434701c2bbd552da8af8b"; }; ``` It does _not_, however, check non-root Nixpkgs inputs for other inputs. If your flake uses [`rust-overlay`][rust-overlay] as an input, for example, then Nix Flake Checker doesn't check `rust-overlay`'s own [Nixpkgs input][rust-overlay-nixpkgs], even though that input is registered in your `flake.lock`. If you used a `follows` statement to pin `rust-overlay`'s Nixpkgs to your root Nixpkgs, however, then Nix Flake Checker _would_ cover that: ```nix { inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable"; # Root Nixpkgs inputs.rust-overlay = { url = "github:oxalica/rust-overlay"; inputs.nixpkgs.follows = "nixpkgs"; # rust-overlay pin to root Nixpkgs }; } ``` ## Run Nix Flake Checker locally Run this command in the same directory as the lock file to check the `flake.lock` in a Nix project: ```shell nix run "github:DeterminateSystems/flake-checker" # Or specify a different location nix run "github:DeterminateSystems/flake-checker" -- /some/other/project/flake.lock ``` This provides a plaintext assessment of your `flake.lock`'s health, indicating any issues with your Nixpkgs inputs and providing pointers on how to bring your flake in line with best practices. You can also add it to your Nix profile: ```shell nix profile install "github:DeterminateSystems/flake-checker" ``` And of course you can add it to a Nix [development environment][dev]. ## Automatic checks using the Nix Flake Checker Action \{#action} While running ad-hoc checks locally and fixing issues is nice, we recommend using automated approaches whenever possible. For [GitHub Actions][actions] users, we've created the [Determinate Flake Checker Action][action]. To add the checker to your pipeline: ```yaml - name: Check Nix flake inputs uses: DeterminateSystems/flake-checker-action@v4 ``` Here's an example of using it in a real pipeline: ```yaml jobs: nix_build: name: Build Nix targets runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Check Nix flake inputs uses: DeterminateSystems/flake-checker-action@main - name: Install Nix uses: DeterminateSystems/determinate-nix-action@v3 - name: Build Nix package run: nix build ``` The Action outputs a [job summary][summary] as [Markdown]. If your `flake.lock` has a clean bill of health, you should see a summary like this: But if your `flake.lock` has issues, you'll see something like this: As you can see, the summary points out not only _what_ is wrong but also how you can fix it. In this case, if you click on **What to do** the summary tells you to use one of a select set of officially supported branches and then shows you how to update your flake inputs accordingly. For the sake of your flakes' long-term health, we recommend using the Nix Flake Checker Action in conjunction with our [update-flake-lock] Action, which automatically submits [`nix flake update`][update] pull requests to flake-backed Nix projects on GitHub. ## Telemetry We'd like you to be aware that Nix Flake Checker collects a bit of anonymized, aggregated telemetry that [Determinate Systems][detsys] uses to measure the impact of our efforts. You can see a full breakdown of what information it collects in the [Nix Flake Checker README][telemetry]. ## Implications At [Determinate Systems][detsys], our goal is to [make Nix better][better]. [Pinning] dependencies is quite possibly _the_ killer feature of [Nix flakes][flakes], and flakes become all the more powerful when they're kept up to date with the most recent work of the incredible Nix community. We believe that supporting best practices in flake dependencies is a small but substantial step forward for developers and orgs that use Nix. We're open to feedback on both [Nix Flake Checker][flake-checker] and the [Nix Flake Checker Action][action], so do feel free to submit issues and pull requests. We welcome bug fixes, feature requests, and everything between. [action]: https://github.com/DeterminateSystems/flake-checker-action [actions]: https://github.com/features/actions [better]: /blog/we-want-to-make-nix-better [cache]: https://zero-to-nix.com/concepts/caching#binary-caches [detsys]: https://determinate.systems [dev]: https://zero-to-nix.com/concepts/dev-env [flake-checker]: https://github.com/DeterminateSystems/flake-checker [flakes]: https://zero-to-nix.com/concepts/flakes [github]: https://github.com [inputs]: https://zero-to-nix.com/concepts/flakes#inputs [markdown]: https://markdownguide.org [nix]: https://nixos.org [nixos-org]: https://github.com/NixOS [nixpkgs]: https://github.com/NixOS/nixpkgs [pinning]: https://zero-to-nix.com/concepts/pinning [rust-overlay]: https://github.com/oxalica/rust-overlay [rust-overlay-nixpkgs]: https://github.com/oxalica/rust-overlay/blob/e75da5cfc7da874401decaa88f4ccb3b4d64d20d/flake.nix#L8 [summary]: https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries [telemetry]: https://github.com/DeterminateSystems/flake-checker#telemetry [update]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-update.html [update-flake-lock]: https://github.com/DeterminateSystems/update-flake-lock --- **Declarative GNOME configuration with NixOS** Published: June 1, 2023 URL: https://determinate.systems/blog/declarative-gnome-configuration-with-nixos I _adore_ tinkering with my machine, trying new tools, extensions, themes, and ideas. When I was younger, it was just a way to learn. Now, it's a way for me to refine my workspace and bring myself small joys. While tinkering can be fun, it can be a chore to set up a new machine, keep configurations up to date between machines, or even just remember to keep up to date backups. What about when we want to configure a whole desktop environment? While [NixOS](https://nixos.org/) offers configuration settings like `services.gnome.gnome-keyring.enable` for system-wide features, there's a lack of knobs when you want to set things like user-specific GNOME 'Favorite Apps' or extensions. Let's explore a useful addition to your NixOS configuration: [Home Manager](https://nix-community.github.io/home-manager/) and its [`dconf`](https://wiki.gnome.org/Projects/dconf) module. > This article uses [Nix flakes](https://nixos.wiki/wiki/Flakes) which is an experimental feature. You may need to set this in your configuration: > > ```nix showLineNumbers > nix.settings.experimental-features = [ "flakes" "nix-command" ]; > ``` ## Getting Home Manager set up Home manager a tool from the Nix ecosystem that helps you take the declarative ideals of Nix/NixOS and apply them to your user's home directory (`$HOME`). It plugs in (as a [NixOS module](https://nixos.wiki/wiki/NixOS_modules)) into an existing NixOS configuration, or can be installed on different Linux as a user service. In order to make some parts of your configuration declarative, Home Manager might take control of certain file paths, or set various options in things like `dconf`. Because of its job, Home Manager can make updating your configuration **feel** more error-prone, but don't fear: Use a VCS like [`git`](https://git-scm.com/) to store your configuration. If your Home Manager setup breaks or acts strange for any reason, check the service status via `systemctl status home-manager-$USER` and `journalctl -u home-manager-$USER.service`. When in doubt, roll back your configuration and delete any files the errors are complaining about. > In our example, the user is named `ana`, and the machine is named `gizmo`. In your Nix flake, add the input for Home Manager and ensure it follows the `nixpkgs` you're using: ```nix showLineNumbers # flake.nix { inputs = { nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; home-manager = { url = "github:nix-community/home-manager"; inputs.nixpkgs.follows = "nixpkgs"; }; # ... }; # ... } ``` If you don't already have GNOME configured, you can do that via a `nixosModule` like this: ```nix showLineNumbers # flake.nix { # ... outputs = { self, nixpkgs, home-manager }: let # ... in { # ... nixosModules = { # ... gnome = { pkgs, ... }: { config = { services.xserver.enable = true; services.xserver.displayManager.gdm.enable = true; services.xserver.desktopManager.gnome.enable = true; environment.gnome.excludePackages = (with pkgs; [ gnome-photos gnome-tour ]) ++ (with pkgs.gnome; [ cheese # webcam tool gnome-music gedit # text editor epiphany # web browser geary # email reader gnome-characters tali # poker game iagno # go game hitori # sudoku game atomix # puzzle game yelp # Help view gnome-contacts gnome-initial-setup ]); programs.dconf.enable = true; environment.systemPackages = with pkgs; [ gnome.gnome-tweaks ] }; }; }; }; } ``` Next, add a `nixosModule` that enables `home-manager`: ```nix showLineNumbers # flake.nix { # ... outputs = { self, nixpkgs, home-manager }: let # ... in { # ... nixosModules = { # ... declarativeHome = { ... }: { config = { home-manager.useGlobalPkgs = true; home-manager.useUserPackages = true; }; }; }; }; } ``` I create a `nixosModule` for each of my users (you may have another way, feel free to do that): ```nix showLineNumbers # flake.nix { # ... outputs = { self, nixpkgs, home-manager }: let # ... in { # ... nixosModules = { # ... users-ana = ./users/ana; }; }; } ``` You can enable Home Manager for your user like this: ```nix showLineNumbers # users/ana/default.nix { ... }: { config = { home-manager.users.ana = ./home.nix; users.users.ana = { # ... }; }; } ``` Now create the `home.nix` referenced above: ```nix showLineNumbers # users/ana/home.nix { ... }: { home.username = "ana"; home.homeDirectory = "/home/ana"; # ... programs.home-manager.enable = true; home.stateVersion = "22.05"; } ``` Before enabling, ensure your `nixosConfiguration` has these modules, as well as `home-manager.nixosModules.home-manager`: ```nix showLineNumbers # flake.nix { # ... outputs = { self, nixpkgs, home-manager }: let # ... in { # ... nixosConfigurations = { gizmo = { system = "aarch64-linux"; modules = with self.nixosModules; [ ({ config = { nix.registry.nixpkgs.flake = nixpkgs; }; }) # ... home-manager.nixosModules.home-manager gnome declarativeHome users-ana ]; }; # ... } }; } ``` With that, you should be able to switch into the new configuration: ```bash nixos-rebuild switch --flake .#gizmo ``` Validate it worked by reviewing the output: ```bash $ systemctl status home-manager-ana.service ● home-manager-ana.service - Home Manager environment for ana Loaded: loaded (/etc/systemd/system/home-manager-ana.service; enabled; preset: enabled) Active: active (exited) since Mon 2022-09-26 22:15:32 PDT; 2s ago Process: 54958 ExecStart=/nix/store/nbhk58wgzvm2w8npi18qzjnn0xjcs3aw-hm-setup-env /nix/store/ig2vhy0pa4rvlkkdc511vyb6plp89x5a-home-manager-generation (code=exited, status=0/SU> Main PID: 54958 (code=exited, status=0/SUCCESS) IP: 0B in, 0B out CPU: 377ms ``` If you see errors, dig deeper via ` journalctl -u home-manager-ana.service`. ## Declaratively configuring GNOME There are a lot of knobs to set in GNOME. GNOME breaks down into having GTK3/4 (which has UI, icon, and cursor themes), as well as an group of fairly tightly integrated components which are primarily configured by `dconf`, which most folks configure via `gnome-settings` or the settings panels of the relevant applications. If you're curious and wanted to watch how/if various `dconf` settings get changed when doing things, you can 'watch' while you click around an application: ```bash $ dconf watch / /org/gnome/control-center/last-panel 'network' /system/proxy/mode 'none' /org/gnome/control-center/last-panel 'background' /org/gnome/desktop/interface/color-scheme 'default' /org/gnome/desktop/interface/color-scheme 'prefer-dark' ``` If it's too noisy, you can limit what you see by changing the `/` to a selector, for example `/org/gnome/desktop/`. Home Manager offers a `dconf` module, which we can use to declaratively set these values. ## GTK3/GTK4 cursor, icon, and window themes To set the GTK icon theme, first search for a theme. [this search](https://search.nixos.org/packages?channel=unstable&from=0&size=200&sort=relevance&type=packages&query=-gtk-theme) or [this one](https://search.nixos.org/packages?channel=unstable&from=0&size=200&sort=relevance&type=packages&query=-theme) can help you find already packaged themes. You can click the "Source" button on any of those packages to see the expression used to package it, just in case you end up needing to make your own. Here are the relevant settings, with some examples of what I've found that I like: ```nix showLineNumbers # users/ana/home.nix { pkgs, ... }: { # ... gtk = { enable = true; iconTheme = { name = "Papirus-Dark"; package = pkgs.papirus-icon-theme; }; theme = { name = "palenight"; package = pkgs.palenight-theme; }; cursorTheme = { name = "Numix-Cursor"; package = pkgs.numix-cursor-theme; }; gtk3.extraConfig = { Settings = '' gtk-application-prefer-dark-theme=1 ''; }; gtk4.extraConfig = { Settings = '' gtk-application-prefer-dark-theme=1 ''; }; }; home.sessionVariables.GTK_THEME = "palenight"; # ... } ``` > Want to set the **GNOME Shell theme?** We do this [below](#gnome-extensions) after discussing `dconf` a bit more. Finding the `name` field for a given package can be a bit inconsistent. 😔 Most of the time, you can guess it, or copy it from what shows up when you check with `gnome-tweaks`. Usually, you can find the package repository via the "Homepage" link on the searches listed above, or in the expression via the `meta.homepage` ([example](https://github.com/NixOS/nixpkgs/blob/62228ccc672ed000f35b1e5c82e4183e46767e52/pkgs/data/themes/gtk-theme-framework/default.nix#L32)) or `src` fields ([example](https://github.com/NixOS/nixpkgs/blob/62228ccc672ed000f35b1e5c82e4183e46767e52/pkgs/data/themes/gtk-theme-framework/default.nix#L7-L9)). From there you can usually find some listing of the theme name ([example](https://github.com/jnsh/arc-theme/blob/b9b98cb394f8acf0a09f3601e3294d406b8e40a5/meson.build#L13-L18)). **To use a theme not already packaged**, you'll need to take a [good starting point](https://github.com/NixOS/nixpkgs/blob/62228ccc672ed000f35b1e5c82e4183e46767e52/pkgs/data/themes/arc/default.nix), edit it, then add your custom package to your flake: ```nix showLineNumbers # flake.nix { # ... outputs = { self, nixpkgs, home-manager }: let supportedSystems = [ "x86_64-linux" "aarch64-linux" ]; forAllSystems = f: nixpkgs.lib.genAttrs supportedSystems (system: f system); # ... in { # ... overlays.default = final: prev: { my-artistanal-theme = final.callPackage ./packages/my-artisanal-theme { }; # ... }; packages = forAllSystems (system: let pkgs = import nixpkgs { inherit system; overlays = [ self.overlays.default ]; # ... }; in { inherit (pkgs) my-artisanal-theme; }; } ``` Once done, you should be able to use it by setting, for example, `gtk.theme.package = pkgs.my-artisanal-theme`. ## Setting GNOME options As mentioned [above](#declaratively-configuring-gnome), most GNOME settings exist in `dconf`. Run `dconf watch /` and set whatever option you're looking to declaratively persist, and observe the output: Here's what I see when I run `gnome-settings` and visit the 'Appearance' pane, then in 'Style' click between 'Light' and 'Dark'. ```bash $ dconf watch / # ... /org/gnome/desktop/interface/color-scheme 'default' /org/gnome/desktop/interface/color-scheme 'prefer-dark' ``` Let's try setting those from the command line an observing the `gnome-settings` window change: ```bash dconf write /org/gnome/desktop/interface/color-scheme "'default'" # Observe `gnome-settings` being light dconf write /org/gnome/desktop/interface/color-scheme "'prefer-dark'" # Observe `gnome-settings` being dark ``` Using this information, we can add this to the user configuration: ```nix showLineNumbers # users/ana/home.nix { pkgs, ... }: { # ... # Use `dconf watch /` to track stateful changes you are doing, then set them here. dconf.settings = { "org/gnome/desktop/interface" = { color-scheme = "prefer-dark"; }; }; } ``` After a bit of tweaking, you might end up with something like this: ```nix showLineNumbers # users/ana/home.nix { pkgs, ... }: { # ... dconf.settings = { # ... "org/gnome/shell" = { favorite-apps = [ "firefox.desktop" "code.desktop" "org.gnome.Terminal.desktop" "spotify.desktop" "virt-manager.desktop" "org.gnome.Nautilus.desktop" ]; }; "org/gnome/desktop/interface" = { color-scheme = "prefer-dark"; enable-hot-corners = false; }; "org/gnome/desktop/wm/preferences" = { workspace-names = [ "Main" ]; }; "org/gnome/desktop/background" = { picture-uri = "file:///run/current-system/sw/share/backgrounds/gnome/vnc-l.png"; picture-uri-dark = "file:///run/current-system/sw/share/backgrounds/gnome/vnc-d.png"; }; "org/gnome/desktop/screensaver" = { picture-uri = "file:///run/current-system/sw/share/backgrounds/gnome/vnc-d.png"; primary-color = "#3465a4"; secondary-color = "#000000"; }; }; } ``` ## GNOME Extensions Once user extensions are enabled, extensions can be added to the `home.packages` set then enabled in `dconf.settings."org/gnome/shell".enabled-extensions`. After, they can be configured just as any other GNOME option as described [just above](#setting-gnome-options). ```nix showLineNumbers # users/ana/home.nix { pkgs, ... }: { # ... dconf.settings = { # ... "org/gnome/shell" = { disable-user-extensions = false; # `gnome-extensions list` for a list enabled-extensions = [ "user-theme@gnome-shell-extensions.gcampax.github.com" "trayIconsReloaded@selfmade.pl" "Vitals@CoreCoding.com" "dash-to-panel@jderose9.github.com" "space-bar@luchrioh" ]; }; }; home.packages = with pkgs; [ # ... gnomeExtensions.user-themes gnomeExtensions.tray-icons-reloaded gnomeExtensions.vitals gnomeExtensions.dash-to-panel gnomeExtensions.sound-output-device-chooser gnomeExtensions.space-bar ]; } ``` A GNOME Shell theme can be picked like this: ```nix showLineNumbers # users/ana/home.nix { pkgs, ... }: { # ... dconf.settings = { # ... "org/gnome/shell" = { disable-user-extensions = false; enabled-extensions = [ "user-theme@gnome-shell-extensions.gcampax.github.com" ]; "org/gnome/shell/extensions/user-theme" = { name = "palenight"; }; }; }; home.packages = with pkgs; [ # ... gnomeExtensions.user-themes palenight-theme ]; } ``` ## Troubleshooting > I made a bunch of changes then ran `nixos-rebuild switch`, but some things didn't change? Log out and log back in. I found some things just didn't set themselves until you did! > I accidentally altered GNOME settings which I'd set via Home Manager, and they aren't changing back on a `nixos-rebuild switch`? It is possible to change `dconf` values once set via Home Manager, if this happens and `nixos-rebuild switch` isn't causing a change, **you may need to restart the `home-manager` service** with `systemctl restart home-manager-$USER`. If that fails, make a change (for example `touch`) your user home configuration first. > Some changes I made are not reflecting when I `nixos-rebuild switch`? Check `systemctl status home-manager-$USER` and ensure the service started successfully, if not, dig in with `journalctl -u home-manager-$USER` and make sure to carefully read the error. > The setting I want isn't tracked by Home Manager or `dconf`? It might be complicated. You can try seeing if the program creates an entry in your XDG directories, such as `~/.config` or `~/.cache`. If so, you can often provision content into the file with the [`home.file`](https://rycee.gitlab.io/home-manager/options.html#opt-home.file) option. Please note this will make the file unwritable, which may impact some programs. ## Conclusion Once various all of your preferred settings are persisted, it becomes straightforward to share key settings and extensions between multiple machines, or even architectures. By taking a little bit more time when configuring our system, we can avoid having to do it again. Using these strategies I was able to have 3 different machines (an x86_64 desktop workstation, an x86_64 laptop, and a headed aarch64 server) with the same settings, and keep them all consistent so I can spend more time doing what I enjoy (tinkering) instead of what I don't (re-configuring machines). If you're looking for a complete home configuration, you can check out mine [here](https://github.com/Hoverbear-Consulting/flake/blob/89cbf802a0be072108a57421e329f6f013e335a6/users/ana/home.nix). Here's what it looks like: --- **Packaging Open Policy Agent policies with Nix** Published: May 22, 2023 URL: https://determinate.systems/blog/open-policy-agent [Open Policy Agent][opa] (**OPA**) is an open source, general-purpose [policy engine][engine] that enables you to express policy logic in a [Datalog]-flavored DSL called [Rego] and evaluate those policies against [JSON]. Policy is of vital importance in many domains of software—complex authorization is probably the most well-known use case—but policy logic can be quite laborious to write in standard programming languages. OPA and Rego provide an elegant alternative to the nested `if`/`else` spaghetti code that we're all painfully familiar with. I've [written about][blog-post], [presented on][presentation], and [contributed to][prs] OPA in the past, so my interest isn't new—in fact, it's been piqued again and again over the years and I'm always looking for excuses to do new things with it. The three things I like most about OPA: - Because it "speaks" JSON, you can use it to evaluate policies in any domain that also speaks JSON, and that now includes just about every domain of software, from [security] to [deployment] to [networking] to [messaging] and beyond. - You can run it in a variety of ways: on the command line using the [`opa`][opa-cli], by POSTing JSON to OPA's [REST API][api], by using it as a [library] (in a [Go] program), and, most promisingly for my use case here, by compiling it to [WebAssembly][wasm] (Wasm) and running it in one of the many places Wasm now runs (imagine evaluating policies using edge workers or in the browser). - Beyond its [concision], [Rego] has a robust [standard library][stdlib], which includes functions for things like [bitwise operations][bitwise], [globs], [regular expressions][regex], [cryptography][crypto], and much more. So with OPA you get both broad applicability and a highly flexible usage model, which to me is quite a formidable pairing. ## An unsatisfied use case By far the most straightforward way to use OPA is with the [OPA CLI][cli]. This command, for example, evaluates `user-info.json` against the [Rego] policy in `rbac-policy.rego`: ```shell opa eval \ --data rbac-policy.rego \ --input user-info.json ``` One drawback of this approach is that OPA needs to be installed in your environment and to have access to both the JSON to be evaluated and the Rego policy. This can make bundling and distribution rather non-trivial in places like continuous integration environments. What I'd like instead is to provide ready-made CLI tools that I can use to evaluate policies without OPA installed or keeping the Rego files locally. In other words, I'd like to fetch a purpose-built binary for a specific policy and provide it with JSON to evaluate, like this: ```shell rbac-evaluate --input-path user-info.json ``` In this post, I'll show you how I used [Nix] to accomplish precisely that—plus some [Rust] in places where I needed a standard programming language to provide the CLI logic. If you want to take a look now, check out the [nix-policy][gh] project on Determinate Systems' [GitHub org][org]. ## What I built nix-policy takes [Rego] policies and uses Nix and Rust to convert them into standalone CLI tools. Let's say you have a Rego policy named `k8s_admission.rego` that provides [admission control][k8s] for a Kubernetes API. You could use nix-policy to generate a CLI tool called `k8s-admission` that provides this help output when you run `k8s-admission --help`: ``` A policy evaluator for the k8s_admission.rego Kubernetes admission control policy Usage: k8s-admission [OPTIONS] Options: -d, --data Policy data object -d, --data-path Path to policy data JSON object -i, --input Policy input object -i, --input-path Path to policy input JSON object -o, --output Result JSON output path -h, --help Print help ``` You could then use `k8s-admission` to evaluate a JSON object encapsulating a Kubernetes admission control request: ```shell k8s-admission \ --input-path ./k8s-admission-request.json \ # The request object --output-path ./evaluation.json # The OPA output object ``` Open Policy Agent itself isn't _directly_ involved here and the `k8s_admission.rego` file doesn't need to be installed on the host at all. We're not shelling out to the `opa` CLI, we're not calling a REST endpoint, and we didn't write Go code to call the OPA Go library. Instead, we use a powerful intermediary: [WebAssembly][wasm]. ## How it works OPA has yet another great feature that I purposely omitted above: you can use it to convert [Rego] policies into [Wasm] binaries. nix-policy uses OPA to convert policies into Wasm and then wraps that Wasm in a Rust CLI tool. The Nix code is a bit complex so I created a convenient function called [`mkPolicyEvaluator`][mkpolicyevaluator] that takes just a few arguments: | Argument | Meaning | | :----------- | :------------------------------------------------------------------------------------------------------- | | `name` | The name of the CLI tool (like `k8s-admission` above) | | `src` | The source root (as in most Nix [derivations][derivation]) | | `policy` | The path to the [Rego] policy file that you want wrapped in the CLI (such as `k8s_admission.rego` above) | | `entrypoint` | The OPA [entrypoint] to the policy output (basically the evaluation root) | The `mkPolicyEvaluator` function uses an internal Nix derivation called [`policyDrv`][policydrv], which calls a [Nushell] script called [`opa-wasm.nu`][opa-wasm]. That script - Uses the [OPA] CLI to build a [WebAssembly][wasm] module called `policy.wasm` (inside a tarball). - [Untars] the contents of the tarball and adds the `policy.wasm` binary to the derivation [output]. - Reduces the size of the binary using [wasm-opt] from [binaryen]. That gives us a WebAssembly binary for a specific policy. Now we need to actually _run_ that binary, which is not so trivial. Fortunately, the good folks at [Matrix] have a [Rust] library called [`rust-opa-wasm`][rust-opa-wasm] that makes this pretty straightforward. It essentially uses the [`wasmtime`][wasmtime] library to provide OPA-built Wasm with properly formed data inputs and then handles the output from the binary—think of it as a kind of membrane. The Rust CLI logic for nix-policy is in the [`eval`][eval] directory. In addition to `rust-opa-wasm` you'll see dependencies on common libraries like [`clap`][clap] and [`serde`][serde]. Pretty standard stuff. But you'll also see some Nix magic in the `eval` package. Check out [this line][policy-name]: ```rust let policy_wasm = tokio::fs::read("%policy%").await?; ``` Why `%policy%` here and not a standard filename? That's because the [`mkPolicyEvaluator`][mkpolicyevaluator] function stores the supplied Rego file [in the Nix store][store]. So we need to supply the Rust code with that path _before_ the code is compiled. Yikes! Well, not really yikes, because [Nixpkgs] has a function called [`substituteInPlace`][substituteinplace] that I use to replace `%policy%` with the right filepath _before_ the CLI is compiled. I do something similar with the [policy name][rego-name] and the [entrypoint][entrypoint-name]. And so Nix enables me to assemble the exact Rust code I want _inside_ the [derivation] that produces the CLI. ### Putting it to use I've provided a few example [policies][Rego] in the repo: - [Role-based access control][rbac-rego] (RBAC) - [Terraform state file][tf-rego] - [A Nix flake checker][flake-rego] Let's run the Nix flake checker on the nix-policy repo's [`flake.lock`][flake]: ```shell git clone https://github.com/DeterminateSystems/nix-policy cd nix-policy nix build .#flake-checker ./result/bin/flake-checker ./flake.bad.lock ``` Uh-oh! ``` ERROR: 2 problems were encountered > Disallowed Git ref for Nixpkgs: some-ancient-ref > Outdated Nixpkgs dependency is 118 days old while the limit is 30 ``` The [`flake.rego`][flake-rego] policy stipulates two things: - If your root [`nixpkgs`][nixpkgs] dependency is based on a specific Git reference, such as `nixpkgs-unstable`, that reference has to be explicitly allowed in the `allowed_refs` array in the [`flake.json`][flake-json] file. In this case, `some-ancient-ref` doesn't make the cut, so the `flake.lock` is rejected. - The root `nixpkgs` dependency must have been updated in the last N days, where N is specified in the `max_days` field of the [`flake.json`][flake-json] data file. Let's take a look at these policies: ```rego package flake # Values from the flake.json data file import data.allowed_refs as allowed_refs import data.max_days as max_days # This keyword hasn't yet landed in stable Rego import future.keywords.in # I rename the input to be a bit more domain specific import input as flake_lock # Helper function has_key(obj, k) { _ = obj[k] } # Deny flake.lock files with a Git ref that's not included in the provided data.json deny[{ "issue": "disallowed-nixpkgs-ref", "detail": { "disallowed_ref": ref, }, }] { has_key(flake_lock.nodes.root.inputs, "nixpkgs") nixpkgs_root := flake_lock.nodes.root.inputs.nixpkgs nixpkgs := flake_lock.nodes[nixpkgs_root] # The root nixpkgs node has_key(nixpkgs.original, "ref") # Check if nixpkgs has an explicit Git ref ref := nixpkgs.original.ref not ref in allowed_refs # Ensure that the ref is explicitly allowed } # Deny flake.lock files where any Nixpkgs was last updated more than 30 days ago deny[{ "issue": "outdated-nixpkgs-ref", "detail": { "age_in_days": floor(age / ((24 * 60) * 60)), "max_days": data.max_days, }, }] { has_key(flake_lock.nodes.root.inputs, "nixpkgs") nixpkgs_root := flake_lock.nodes.root.inputs.nixpkgs nixpkgs := flake_lock.nodes[nixpkgs_root] # The root nixpkgs node last_mod := nixpkgs.locked.lastModified age := (time.now_ns() / 1000000000) - last_mod secs_per_max_period := max_days * ((24 * 60) * 60) age > secs_per_max_period } ``` And there we have it! A Rego policy for `flake.lock` files transformed into a single-purpose CLI tool that can run on any Linux or macOS system. In principle, you could apply a much broader range of policies to Nix flakes, including explicit allowlists and denylists for dependencies, Git refs, and Git repositories. ## Be wary of size As always in software, there's a downside to this approach. The CLIs that nix-policy generates are pretty bulky, usually over 10 MB. But it's important to keep in mind that each CLI bundles its own [Wasm runtime][wasmtime] and that you don't need to have OPA—a bulky dependency in itself—available on your system at all (it's already done its work during the build phase). So these aren't exactly lean and nimble CLIs but I think they're reasonably sized given the work they do and the dependencies that they make unnecessary. ## Conclusion Overall, this was a highly fruitful exploration of Nix and its ability to package software in novel ways. I wasn't sure that turning Rego policies into CLIs would be possible at the outset but at every turn Nix provided the levers I needed to get over the hump. I'll be writing about more adventures using Nix to package WebAssembly here on the blog soon, so stay tuned! [api]: https://www.openpolicyagent.org/docs/latest/rest-api [binaryen]: https://github.com/WebAssembly/binaryen [bitwise]: https://www.openpolicyagent.org/docs/latest/policy-reference/#bits [blog-post]: https://blog.openpolicyagent.org/policy-driven-continuous-integration-with-open-policy-agent-b98a8748e536 [clap]: https://github.com/clap-rs/clap [cli]: https://github.com/DeterminateSystems/nix-policy/blob/main/nix/nu/opa-wasm.nu#L16 [concision]: https://www.openpolicyagent.org/docs/latest/policy-language/#the-basics [crypto]: https://www.openpolicyagent.org/docs/latest/policy-reference/#crypto [datalog]: https://en.wikipedia.org/wiki/Datalog [deployment]: https://www.openpolicyagent.org/docs/latest/terraform [derivation]: https://zero-to-nix.com/concepts/derivations [engine]: https://www.techtarget.com/searchitoperations/definition/policy-engine [entrypoint]: https://www.openpolicyagent.org/docs/latest/management-bundles/#bundle-file-format [entrypoint-name]: https://github.com/DeterminateSystems/nix-policy/blob/main/eval/src/main.rs#L35 [eval]: https://github.com/DeterminateSystems/nix-policy/tree/main/eval [flake]: https://zero-to-nix.com/concepts/flakes [flake-json]: https://github.com/DeterminateSystems/nix-policy/blob/main/examples/flake.json [flake-rego]: https://github.com/DeterminateSystems/nix-policy/blob/main/examples/flake.rego [gh]: https://github.com/DeterminateSystems/nix-policy [globs]: https://www.openpolicyagent.org/docs/latest/policy-reference/#glob [go]: https://go.dev [json]: https://json.org [k8s]: https://www.openpolicyagent.org/docs/latest/kubernetes-primer [library]: https://pkg.go.dev/github.com/open-policy-agent/opa/rego [matrix]: https://matrix.org [messaging]: https://www.openpolicyagent.org/docs/latest/kafka-authorization [mkpolicyevaluator]: https://github.com/DeterminateSystems/nix-policy/blob/main/nix/evaluator.nix [networking]: https://www.openpolicyagent.org/docs/latest/envoy-introduction [nix]: https://nixos.org [nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs [nushell]: /blog/nuenv [opa]: https://openpolicyagent.org [opa-cli]: https://www.openpolicyagent.org/docs/latest/cli [opa-wasm]: https://github.com/DeterminateSystems/nix-policy/blob/main/nix/nu/opa-wasm.nu [org]: https://github.com/DeterminateSystems [output]: https://github.com/DeterminateSystems/nix-policy/blob/main/nix/nu/opa-wasm.nu#L36 [policy-name]: https://github.com/DeterminateSystems/nix-policy/blob/main/eval/src/main.rs#L22 [policydrv]: https://github.com/DeterminateSystems/nix-policy/blob/main/nix/evaluator.nix#L24-L28 [presentation]: https://www.meetup.com/colorado-kubernetes-cloud-native/events/262404329 [prs]: https://github.com/open-policy-agent/opa/pulls?q=is%3Apr+is%3Aclosed+author%3Alucperkins [rbac-rego]: https://github.com/DeterminateSystems/nix-policy/blob/main/examples/rbac.rego [rego-name]: https://github.com/DeterminateSystems/nix-policy/blob/main/eval/src/main.rs#L53 [regex]: https://www.openpolicyagent.org/docs/latest/policy-reference/#regex [rego]: https://www.openpolicyagent.org/docs/latest/policy-language [rust]: https://rust-lang.org [rust-opa-wasm]: https://github.com/matrix-org/rust-opa-wasm.git [security]: https://www.openpolicyagent.org/docs/latest/ssh-and-sudo-authorization [serde]: https://serde.rs [stdlib]: https://www.openpolicyagent.org/docs/latest/policy-reference [store]: https://zero-to-nix.com/concepts/nix-store [substituteinplace]: https://nixos.org/manual/nixpkgs/stable/#fun-substituteInPlace [tf-rego]: https://github.com/DeterminateSystems/nix-policy/blob/main/examples/tfstate.rego [untars]: https://github.com/DeterminateSystems/nix-policy/blob/main/nix/nu/opa-wasm.nu#L28 [wasm]: https://webassembly.org [wasm-opt]: https://github.com/WebAssembly/binaryen#tools [wasmtime]: https://wasmtime.dev/ --- **Extending NixOS configurations** Published: March 31, 2023 URL: https://determinate.systems/blog/extending-nixos-configurations [NixOS][nixos] [modules][nixos-modules] and configurations offer us a tantalizing way to express and share systems. My friends and I can publish our own flakes containing `nixosModules` and/or `nixosConfigurations` outputs which can be imported, reused, and remixed. When it comes to secret projects though, that openness and ease of sharing can be a bit of a problem. Let's pretend my friend wants to share a secret NixOS module or package with me, they've already given me access to the GitHub repository, and now I need to add it to my flake. My [flake][hoverbear-consulting-flake] is public and has downstream users. I can't just up and add it as an input. For one thing, it'd break everything downstream. More importantly, my friend asked me not to. It's terribly inconvenient to add the project as an input and be careful to never commit that change to the repository. Worse, if I did screw up and commit it, my friend might be disappointed in me. We just can't have that. Let's explore how to create a flake which extends some existing flake, a pattern which can be combined with a `git+ssh` flake URL to resolve this precarious situation. This same situation strategy can be applied to other outputs of a flake, and can be combined with [`nixpkgs.lib.makeOverridable`](https://nixos.org/manual/nixpkgs/stable/#sec-lib-makeOverridable). ## Extending a Flake Let's assume for a moment we have a small flake called `original` with a `nixosConfiguration` and a `nixosModule`: ```nix showLineNumbers # original/flake.nix { inputs = { nixpkgs.url = "github:nixos/nixpkgs/nixos-22.11"; }; outputs = { self, nixpkgs }: { nixosModules.default = { config, pkgs, lib, ... }: { # Create a file `/etc/original-marker` environment.etc.original-marker.text = "Hello!"; # You can ignore these, just keeping things small boot.initrd.includeDefaultModules = false; documentation.man.enable = false; boot.loader.grub.enable = false; fileSystems."/".device = "/dev/null"; system.stateVersion = "22.11"; }; nixosConfigurations.teapot = nixpkgs.lib.nixosSystem { system = "x86_64-linux"; modules = [ self.nixosModules.default ]; }; }; } ``` While our little demo configuration, `teapot`, might not be a practical system for hardware (or even a VM) it's a perfectly valid NixOS expression which we can build and inspect the resultant output filesystem of: ```bash extension-demo/original ❯ nix build .#nixosConfigurations.teapot.config.system.build.toplevel extension-demo/original ❯ cat result/etc/original-marker 6ello!⏎ ``` Now, let's assume there exists some other flake, called `extension` that looks like this: ```nix showLineNumbers # extension/flake.nix { inputs = { nixpkgs.url = "github:nixos/nixpkgs/nixos-22.11"; }; outputs = { self, nixpkgs }: { nixosModules.default = { config, pkgs, lib, ... }: { # Create a file `/etc/extension-marker` environment.etc.extension-marker.text = "Hi!"; }; }; } ``` Our goal is to create a `teapot` with the `extension.nixosModules.default` module included. To do that, we'll create a new flake, called `extended` which looks like this: ```nix showLineNumbers # extended/flake.nix { inputs = { original.url = "/home/ana/git/extension-demo/original"; nixpkgs.follows = "original/nixpkgs"; extension = { url = "/home/ana/git/extension-demo/extension"; inputs.nixpkgs.follows = "original/nixpkgs"; }; }; outputs = { self, nixpkgs, original, extension }: original.outputs // { nixosConfigurations.teapot = original.nixosConfigurations.teapot.extendModules { modules = [ extension.nixosModules.default ]; }; }; } ``` Because of `outputs = { self, nixpkgs, original, extension }: original.outputs // { /* ... */ }` this `extended` flake has the same outputs of `original`, plus whatever overrides we add inside `{ /* ... */ }`. ``` extension-demo/extended ❯ nix flake show path:/home/ana/git/extension-demo/extended?lastModified=1680125924&narHash=sha256-EV4jQJ5H3mypuOt4H174lII2yhnaUbZ9rbML2mjyRlI= ├───nixosConfigurations │ └───teapot: NixOS configuration └───nixosModules └───default: NixOS module ``` We call [`extendModules`][github-nix-extendModules] on the original `nixosConfiguration.teapot` to extend the configuration with new modules, in this case, our `extension.nixosModules.default`. Now we can inspect the resultant output filesystem: ```bash extension-demo/extended ❯ nix build .#nixosConfigurations.teapot.config.system.build.toplevel extension-demo/extended took 2s ❯ cat result/etc/original-marker Hello!⏎ extension-demo/extended ❯ cat result/etc/extension-marker Hi!⏎ ``` ## Using private GitHub inputs in flakes While the `github:username/repository` flake paths utilize Github's API and work well for public repositories, you may experience issues trying to use it with a private repository. In these cases, try using the `git+ssh` protocol. For example: ```nix showLineNumbers { inputs = { original.url = "/home/ana/git/extension-demo/original"; nixpkgs.follows = "original/nixpkgs"; extension = { url = "git+ssh://git@github.com/AmbiguousTechnologies/top-secret-project.git"; inputs.nixpkgs.follows = "original/nixpkgs"; }; }; outputs = { self, nixpkgs, original, extension }: original.outputs // { nixosConfigurations.teapot = original.nixosConfigurations.teapot.extendModules { modules = [ extension.nixosModules.default ]; }; }; } ``` Running `nix flake update` should then use your SSH key and work, as long as you have the ability to clone that repository over SSH. [hoverbear-consulting-flake]: https://github.com/Hoverbear-Consulting/flake [nixos-modules]: https://nixos.wiki/wiki/NixOS_modules [nixos]: https://nixos.org/ [github-nix-extendModules]: https://github.com/NixOS/nixpkgs/blob/4e416a8e847057c49e73be37ae8dc4fcdfe9eff8/lib/modules.nix#L333-L354 --- **Nuenv: an experimental Nushell environment for Nix** Published: March 28, 2023 URL: https://determinate.systems/blog/nuenv I appreciate [Bash](https://www.gnu.org/software/bash/) for its stability and its near-ubiquity in Unix environments, but it has some clear drawbacks. I constantly get bitten by things like variable handling and string interpolation, I almost always need to supplement my Bash scripts with tools like `awk`, `sed`, `curl`, `jq`, and `wget` to get even basic things done, and I'd be hard-pressed to use "Bash" and "elegant" or "expressive" in the same sentence. Despite shortcomings like these, [Nixpkgs](https://zero-to-nix.com/concepts/nixpkgs) uses [Bash](https://www.gnu.org/software/bash) as the builder in its [standard environment](https://nixos.org/manual/nixpkgs/stable/#chap-stdenv) (or `stdenv`), most notably in the `stdenv.mkDerivation` function. This wrapper over Nix's built-in derivation function is used almost everywhere in [Nixpkgs](https://zero-to-nix.com/concepts/nixpkgs) and acts as a de-facto default in the Nix community. I believe that standardizing on Bash as a known quantity was absolutely the right choice for the standard environment. But I've always been intrigued by the possibility of using a more powerful, ergonomically-minded shell to [realise](https://zero-to-nix.com/concepts/realisation) Nix derivations. So I decided to give it a try by creating a Nix realisation environment that uses [Nushell](https://www.nushell.sh/), a powerful, cutting-edge shell written in [Rust](https://rust-lang.org) and under heavy development, instead of Bash. I'm calling it [Nuenv](https://github.com/DeterminateSystems/nuenv) and it's proven to be quite a fruitful exercise and yielded some promising initial results. ## Why I chose Nushell There are many "alt" shells out there, like [Fish](https://fishshell.com/), [Elvish](https://elv.sh/), and [Oil](https://www.oilshell.org/). These are all exciting projects but Nushell really stands out to me. Here's a non-exhaustive list of reasons why: - It offers built-in support for [JSON](https://www.nushell.sh/commands/docs/from_json.html), [CSV](https://www.nushell.sh/commands/docs/from_csv.html), [YAML](https://www.nushell.sh/commands/docs/from_yaml.html), [TOML](https://www.nushell.sh/commands/docs/from_toml.html), [SQLite](https://www.nushell.sh/book/loading_data.html#sqlite), and even [DataFrames](https://www.nushell.sh/book/dataframes.html). - It offers a broad range of built-in [commands](https://www.nushell.sh/commands/) that play nicely together and more or less replace tools like `jq`, `curl`, `awk`, `cut`, `join`, and `sed`. - It enables you to create [custom commands](https://www.nushell.sh/book/custom_commands.html#custom-commands) with typed inputs and auto-generated (and pretty) help output. - It supports nested data [records](https://www.nushell.sh/book/types_of_data.html#records), which gives it a kind of object-oriented flavor. - It has built-in support for [testing](https://www.nushell.sh/book/testing.html). - It offers scoping mechanisms like [modules](https://www.nushell.sh/book/modules.html) and [overlays](https://www.nushell.sh/book/overlays.html). While Nushell isn't _quite_ a general-purpose programming language, it certainly skirts the boundary between a shell and a full-fledged language. Not all of Nushell's features are equally germane to an alternative environment for Nix, of course, but I think that these features show how ambitious the project is. Here's what I think Nushell has to offer a new Nix environment: - It makes a clean separation between environment and standard variables. `$foo` would refer to a variable in the local scope but only `$env.foo` could refer to an environment variable. This enables you to avoid those un-fun variable name clashes that pop up so often in Bash. - [String handling](https://www.nushell.sh/commands/categories/strings.html) is powerful enough to obviate the need for the usual smorgasbord of tools required for string parsing, like `awk`, `sed`, and `tr`. - The [custom commands](https://www.nushell.sh/book/custom_commands.html#custom-commands) - with [typed parameters](https://www.nushell.sh/book/custom_commands.html#parameter-types) that I mentioned above make the environment much more robust than Bash's "stringly" typing and awkward function argument handling. Here's an example custom command: ``` # Say hello to and add a ! to the end if is set. def sayHello [ name: string, # The person to say hello to --exclaim(-e): bool, # Whether to add a ! at the end ] { echo $"Hello, ($name)(if $exclaim { "!" })" } ``` Now here's the help output you get by running `sayHello --help`: ``` Say hello to and add a ! to the end if is set. Usage: > sayHello {flags} Flags: -e, --exclaim - Whether to add a ! at the end -h, --help - Display the help message for this command Parameters: name : The person to say hello to ``` As you can see, custom commands in Nushell feel like well-crafted CLI tools in their own right. This makes Nushell-based environments far more introspectable than their Bash counterparts. ## How I created the environment To create my new build environment for Nix, [Nuenv](https://github.com/DeterminateSystems/nuenv), I first wrote a Nix function wrapping Nix's built-in [derivation](https://zero-to-nix.com/concepts/derivations) function: ```nix showLineNumbers { mkNushellDerivation = { nushell, # A Nushell package name, # The name of the derivation src, # The derivation's sources system, # The host system packages ? [], # Same as in stdenv build ? "", # Same as in stdenv }: derivation { inherit build name packages src system; builder = "${nushell}/bin/nu"; args = [ ../nuenv/bootstrap.nu ]; # Attributes passed to the environment # (prefaced with __nu_ to avoid naming collisions) __nu_builder = ../nuenv/builder.nu; __nu_nushell_version = nushell.version; __nu_envFile = ./env.nu; # Helper functions }; } ``` The basic mechanics: Nushell (set as the builder instead of Bash) runs a script called [`bootstrap.nu`](https://github.com/DeterminateSystems/nuenv/blob/0fbc1e0d6a398335e8404b0039c611fe462bc081/nuenv/bootstrap.nu) that performs some setup actions (like making the Nushell executable itself discoverable in the env). The bootstrapper then runs a [`builder.nu`](https://github.com/DeterminateSystems/nuenv/blob/0fbc1e0d6a398335e8404b0039c611fe462bc081/nuenv/builder.nu) script that performs the actual realisation. As in all Nix derivations, the other attributes of the derivation are set as environment variables available to that script (plus some others that Nix provides). The `builder.nu` script needs to do quite a lot, including: - Creating the directory in the [Nix store](https://zero-to-nix.com/concepts/nix-store) where build output will end up (Nix provides an `out` environment variable for that but it doesn't automatically create that directory). - Copying all of the derivation's sources (`src` or `srcs` in the standard environment) into the build sandbox, which resides in a temporary directory. Nix _does_ put those sources in the Nix store at `/nix/store/$HASH-source` but the builder script needs to copy them into the sandbox. - Making sure that the sandbox can discover any packages required by the derivation process using the `PATH`. That establishes a kind of bare minimum environment. Beyond that, a builder script _should_ provide things like helper functions (_commands_ in Nushell) for use in derivation logic. The standard environment provides many such functions via the [`user-env.nu`](https://github.com/DeterminateSystems/nuenv/blob/24c11410b359f8f96572cf97cb6e67177fad3200/nuenv/user-env.nu) script, like [`wrapProgram`](https://nixos.org/manual/nixpkgs/stable/#fun-wrapProgram) and [`substituteInPlace`](https://nixos.org/manual/nixpkgs/stable/#fun-substituteInPlace). For Nuenv, I've provided two so far: - [`substitute`](https://github.com/DeterminateSystems/nuenv/blob/0fbc1e0d6a398335e8404b0039c611fe462bc081/nuenv/user-env.nu#L60-L77), which performs string substitution in a specified file and writes the output to a new file. - [`substituteInPlace`](https://github.com/DeterminateSystems/nuenv/blob/0fbc1e0d6a398335e8404b0039c611fe462bc081/nuenv/user-env.nu#L79-L94) is like `substitute` but rewrites the specified file in place. In principle, however, the full battery of `stdenv` functions could be replaced with custom Nushell commands. Overall, I was extremely impressed with how elegant the Nushell build script is compared to what its Bash equivalent would be. I'll provide an illustrative example. Here's how I [set the](https://github.com/DeterminateSystems/nuenv/blob/0fbc1e0d6a398335e8404b0039c611fe462bc081/nuenv/builder.nu#L82-L87) [`PATH`](https://github.com/DeterminateSystems/nuenv/blob/0fbc1e0d6a398335e8404b0039c611fe462bc081/nuenv/builder.nu#L82-L87) to discover packages in the Nix store: ``` # Parse the __nu_packages environment variable string, where each # package path is separate by a space, and convert to a list let packages = ($env.__nu_packages | split row (char space)) # Convert the list to a colon-separated string let $packagesPath = ( $packages | each { |pkg| $"($pkg)/bin" } | str collect (char esep) ) # Set the PATH let-env PATH = $packagesPath ``` Here's how you would accomplish something like that in Bash: ```bash export PATH= for i in $packages; do if [ "$i" = / ]; then i=; fi PATH=$PATH${PATH:+:}$i/bin done ``` As you can see, Nushell feels closer to something like Ruby. Despite wins like this, Nuenv in its current state does have some major shortcomings vis-à-vis the Nixpkgs standard environment: - There's only one realisation phase, called `build`. By contrast, the stdenv is much more nuanced, offering support for Makefiles, `configure` scripts, and differentiated `phases` like `buildPhase`, `installPhase`, `configurePhase`, and `checkPhase`. - You can only pass one set of packages to the environment, whereas stdenv makes a [distinction](https://discourse.nixos.org/t/use-buildinputs-or-nativebuildinputs-for-nix-shell/8464) between `buildInputs`, `nativeBuildInputs`, `propagatedBuildInputs`, and others. The dependencies you supply using packages in Nuenv are only available at build time and aren't included in the runtime package, while stdenv enables you to target dependencies to both build time and run time as well as things like cross-compilation (which is not supported in Nuenv). - Nuenv-based realisations are likely to be a bit more costly, in terms of build time and disk space used by the Nix store, than `stdenv.mkDerivation` because the Nushell package is required, although caching and the fact that Nushell doesn't require [coreutils](https://www.gnu.org/software/coreutils/) should make this cost relatively small. A related limitation is that you can only realise Nuenv-based derivations on systems that can run Nushell. While that should be a fairly sizable range of systems given that Nushell is built in [Rust](https://rust-lang.org/), it's nowhere near the range of systems that can run Bash. So definitely don't use Nuenv for any serious purpose just yet. ## User-facing benefits of Nuenv Although Nuenv is still at an early stage, I think that it already has some nice advantages over stdenv: - Inside your derivation logic, you get access to the broad Nushell feature set I talked about above. - It has much prettier log output thanks to Nushell's [ANSI](https://www.nushell.sh/commands/docs/ansi.html) support. While this is more of a quality-of-life thing, it does liven things up a bit. - You can try out the Nuenv environment locally by running `nix develop .#nuenv` if you clone the repo (or `nix develop github:DeterminateSystems/nuenv#nuenv` if you don't want to clone the repo locally). This gives you access to Nuenv's helper commands. Run `substituteInPlace --help` to see an example. As far as I know, there isn't really a way to introspect the stdenv in this way. You _can_ access Bash with stdenv's functions provided by running `nix-shell -p` but those functions don't provide help output. You can also see a list of custom commands available to Nuenv derivations by running `nix run github:DeterminateSystems/nuenv#nuenv-commands`. ## Try it out You can try out Nuenv in several ways. To build an example package using the environment: ```bash nix build --print-build-logs "github:DeterminateSystems/nuenv" # See the result cat result/share/hello.txt ``` This is a pretty basic derivation that pipes a string to [GNU hello](https://www.gnu.org/software/hello/) and writes that output to the `share` directory in the build output. You can also build your own package with Nuenv! Here's an example `flake.nix`: ```nix showLineNumbers { inputs = { nixpkgs.url = "nixpkgs"; nuenv.url = "github:DeterminateSystems/nuenv"; }; outputs = { self, nixpkgs, nuenv }: let overlays = [ nuenv.overlays.default ]; systems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ]; forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f { inherit system; pkgs = import nixpkgs { inherit overlays system; }; }); in { packages = forAllSystems ({ pkgs, system }: { default = pkgs.nuenv.mkDerivation { name = "hello"; src = ./.; # This script is Nushell, not Bash! build = '' "Hello" | save hello.txt let out = $"($env.out)/share" mkdir $out cp hello.txt $out ''; }; }); }; } ``` If you've [installed Nix with flakes enabled](https://zero-to-nix.com/start/install), you can copy this flake.nix into a directory and run `nix build && cat ./result/share/hello.txt` to see the result. To run a Nushell script instead of providing a raw string, you can supply `build = builtins.readFile ./my-nushell-script.nu` to the derivation. ## Ramifications I think that using a more powerful builder for Nix—maybe Nushell, maybe something else—has major implications: - If can offer a **dramatically improved developer experience** around derivation. Right now, derivation remains a bit of a dark art and one of the more intimidating aspects of using Nix (which is already intimidating in itself). - It can **radically alter the division of labor between Nix and the builder**. Because Bash isn't terribly powerful, Nix has to do a lot of the work of making things palatable to Bash, like constructing complex strings. But with a more powerful shell, Nix can defer a lot of work to the builder. Wouldn't it be nice to just use Nix to supply attribute sets and let the builder handle the rest? - Because any Nix environment is just Nix, you could **introduce something like Nushell piecemeal into the Nix dependency tree**. Imagine replacing some core utilities in Nixpkgs with more robust, ergonomic equivalents or dramatically simplifying language-specific derivation wrappers. While I don't see Nushell or its ilk taking over the Nix world by storm any time soon, I do hope that this post gets you thinking about what a qualitatively better builder might mean for Nix. ## Conclusion: an oh-so-worthy experiment My work on [Nuenv](https://github.com/DeterminateSystems/nuenv) hasn't yielded a particularly robust Nix environment just yet but it's been a fantastic learning experience about some of the lower-level nitty gritty of how Nix works. If you're looking to level up your Nix knowledge, I strongly encourage you to take on a similar project (or hack on Nuenv!). You may just end up blazing a fruitful new trail for yourself and others in the Nix community—or with a renewed appreciation for the tried-and-true standard environment. Finally, using Nushell in conjunction with Nix has frankly sparked joy in a way that trusty old Bash just doesn't (for me, at least). While that isn't a "engineeringly" justification for building something, there's much to be said for working on things that compel a dramatic shift of energy and attention. We don't have any concrete plans to take Nuenv much further at this time but we'd love to see how the community reacts. --- **Moving stuff around with Nix** Published: March 22, 2023 URL: https://determinate.systems/blog/moving-stuff-around-with-nix Nix has an extremely broad feature set that enables you to do a lot of fun stuff. In past posts here on the Determinate Systems [blog](/blog) I've touted some of the more widely used features, such as [reproducible development environments](/blog/nix-direnv) and [declarative home environments](/blog/nix-home-env), but today I want to focus on something less widely used: `nix copy` (or [`nix-copy-closure`](https://nixos.org/manual/nix/stable/command-ref/nix-copy-closure.html) if you're using the older, non-unified Nix CLI). This utility enables you to copy what are called Nix [_closures_](https://zero-to-nix.com/concepts/closures) between machines, which can be quite handy in a variety of distribution and deployment scenarios. Here's an example `nix copy` command: ```bash nix copy \ --to ssh-ng://my-remote-host \ "nixpkgs#apachakafka_3_3" ``` This command would copy, via SSH, everything required to run [Apache Kafka](https://kafka.apache.org/) v3.3 to a remote host with Nix installed. And as we'll see in this post, the "everything required" is what sets `nix copy` apart from other tools. ## Closures in Nix In Nix, a package's [_closure_](https://zero-to-nix.com/concepts/closures) encapsulates everything needed to either build or run the package: - _Build closures_ include all the dependencies required to build the package, such as compilers, package managers, and shell utilities. - _Runtime closures_ include everything required to run any or all programs in the package, such as configuration files, dynamically linked libraries, or other programs (imagine a shell utility that relies on output from `openssl` or `ffmpeg` to work). Which closure type you need to work with depends on the use case at hand, but runtime closures are, unsurprisingly, more common in deployment scenarios (as in the project I'll talk about below). By contrast, most tools that move stuff between machines, such as [rsync](https://linux.die.net/man/1/rsync) and [scp](https://linux.die.net/man/1/scp), typically "think" in terms of files and directories and are thus fundamentally _procedural_. If you need to copy a dependency tree to another machine you need to do so manually, which, as many of us have learned the hard way, is quite error prone. But `nix copy` thinks in terms of full closures, which are a fundamentally _declarative_ abstraction. You tell Nix the "what" (the package to copy) and the entire dependency tree comes along with it. The "how" of this operation is baked into Nix, which pretty much _can't_ think in terms of plain old files and directories. And because `nix copy` interacts with the [Nix store](https://zero-to-nix.com/concepts/nix-store), it can use the core mechanics of the [Nix store](https://zero-to-nix.com/concepts/nix-store) and [caching](https://zero-to-nix.com/concepts/caching) to make the copy operation efficient. On any given `nix copy` run you may need to build an entire closure from scratch or you may be able to get most of the closure from the target machine's Nix store or a configured binary cache. If the target machine already has some dependencies in the Nix store, `nix copy` is essentially a no-op for those. ## Why `nix copy` There are many potential use cases for `nix copy`, but the one with the most clear value is maintaining a clean separation between machines that build packages and machines that use them. You can, for example, run a beefy CI machine that copies a Nix package that's highly resource intensive to build onto less beefy machines that then use the package. In this example, a beefy CI machine copies a package called `service-pkg` defined in a [local flake](https://zero-to-nix.com/concepts/flakes) to a VM in the cloud: ```bash # On the beefy CI machine nix copy \ --to ssh-ng://"${CLOUD_VM_IP}" \ ".#packages.x86_64-linux.service-pkg" ``` Another option would be to stand up a cloud VM and copy a closure from a beefy CI machine: ```bash nix copy \ --from ssh-ng://"${CI_MACHINE_IP}" \ ".#packages.x86_64-linux.service-pkg" ``` By default, Nix copies the runtime closure, but if for some reason we needed to copy the build closure we could apply the `--derivation` flag: ```bash nix copy \ --derivation \ --to ssh-ng://"${CLOUD_VM_IP}" \ ".#packages.x86_64-linux.service-pkg" ``` ## Example project To demonstrate `nix copy` in action, I created a fairly straightforward example project: [https://github.com/DeterminateSystems/nix-copy-deploy](https://github.com/DeterminateSystems/nix-copy-deploy) In this project, I use [Terraform](https://github.com/DeterminateSystems/nix-copy-deploy/blob/main/main.tf) to stand up some Linux droplets on [Digital Ocean](https://digitalocean.com/). Once the droplets have been created, I run a [shell script](https://github.com/DeterminateSystems/nix-copy-deploy/blob/main/scripts/deploy.sh) that reads the [Terraform state file](https://developer.hashicorp.com/terraform/language/state) to gather a list of IPs for the droplets and then runs a few commands on each droplet via SSH: - It [installs Nix](https://github.com/DeterminateSystems/nix-copy-deploy/blob/main/scripts/deploy.sh#L7) using the [Determinate Nix Installer](https://github.com/determinateSystems/nix-installer), an experimental tool that we at Determinate Systems [recently released](/blog/determinate-nix-installer). - It [nix copy](https://github.com/DeterminateSystems/nix-copy-deploy/blob/main/scripts/deploy.sh#L41-L43) a Nix package defined in the [local flake](https://github.com/DeterminateSystems/nix-copy-deploy/blob/main/flake.nix#L19)s onto the droplet (specifically a program called [ponysay](https://github.com/erkin/ponysay). - It feeds the text `Hello from nix copy!` to ponysay, which then outputs a lovely greeting from a cartoon horse. The command that accomplishes this: ```bash nix copy \ --to ssh-ng://root@"${DROPLET_IP}" \ ".#packages.x86_64-linux.ponysay" ``` Standing up cloud VMs to run ponysay isn't exactly a show-stopping use case, but I think it illustrates the mechanics of `nix copy`. You could replace ponysay here with _any_ package output from any Nix flake: it could be a web service or Postgres or Kafka or even an entire NixOS configuration. If you can package it using Nix—and that's pretty much anything imaginable—then you can `nix copy` it onto a machine with Nix installed and run it. And unlike with many other deployment approaches, with `nix copy` you don't have to fuss with rsync and shell scripts. You tell the Nix CLI what needs to be copied and Nix handles the how, no matter how intricate, of shuttling it to its destination. ## Other features ### S3 support `nix copy` supports storage systems compatible with the [S3 API](https://aws.amazon.com/s3/). In this example, Nix copies a package closure defined in a local flake to a bucket in AWS Singapore: ```bash nix copy \ --to "s3://my-nix-stuff-bucket?region=ap-southeast-1" \ ".#my-package" ``` ### Security When you build or copy closures with Nix, you have the option of using _substituters_, which are [binary caches](https://zero-to-nix.com/concepts/caching#binary-caches) that enable you to fetch already-built dependencies rather than building them anew. You can specify substituters (by URL) either per Nix CLI command invocation or in your Nix configuration. Binary caches can be public (available to all) or private. To use private caches, you need to supply public keys. The `nix copy` utility verifies that all paths are signed by these public keys when using substituters, which ensures that no one can push artifacts to your cache without a valid key. You can, however, explicitly disable public key checking with the [--no-check-sigs](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-copy#options) flag. This doesn't make private binary caches bulletproof, of course, but it does tighten up the supply chain. ## Nix in multi-machine contexts While Nix is fantastic for deterministic package builds, reproducible development environments, and much more, utilities like `nix copy` propel Nix beyond the "usual" capabilities we associate with package managers and unlock use cases suited for larger orgs and heterogeneous environments. If you've found this post a little hum-drum, we're okay with that. `nix copy` is kind of, well, _boring_. You give it something to copy, you specify a source and a destination, and that's it. But there's also a certain beauty to that—a beauty that may become more clear to you next time you're debugging a mysteriously failed rsync run. --- **Introducing the Determinate Nix Installer** Published: February 27, 2023 URL: https://determinate.systems/blog/determinate-nix-installer We at Determinate Systems are extremely excited to announce the release of the [Determinate Nix Installer](https://github.com/DeterminateSystems/nix-installer), a brand-new installer for [Nix](https://nixos.org/). You can run the installer on a [variety of systems](https://github.com/determinateSystems/nix-installer#the-determinate-nix-installer), including macOS, Linux, Windows via WSL2, and more, with this command: ```shell curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install ``` Check out the [Zero to Nix quick start](https://zero-to-nix.com/start/install) for a more in-depth look. In this post, we'll cover how our new installer improves on the current official installer, why and how we built it, and how we envision its future. ## Why installing Nix is a complex problem Sometimes, installing software is straightforward. You fetch a binary from a URL, you copy it to a directory on your `PATH`, and you're good to go. But sometimes things aren't nearly so pleasant. [Nix](https://nixos.org), as powerful and transformative as it is, falls firmly in the "not nearly so pleasant" camp. Far from just a static binary that you can fetch, copy, and call it a day, Nix requires a broad set of changes to your system, from creating new users to installing and running a [daemon](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-daemon.html) to creating a [root volume](https://zero-to-nix.com/concepts/nix-store) and beyond. Nix installation is a non-trivial problem—and one that by definition can't be solved by Nix itself. Currently, the [official installer](https://nixos.org/download.html) for Nix is a Bash script that does the job. It's been used with success many thousands of times in various settings. Some of our team members have even worked on the official installation script. It's widely used because it **_works_**. But for reasons that we'll lay out here, we've felt for some time that the official installer has shortcomings that make Nix onboarding more challenging than it needs to be. ## A fresh start From our perspective, the current official installer has three main problems: - It's written in Bash. We love Bash here at Determinate Systems, but we think it's unsuitable for something as fine-grained as installing Nix. Different Bash implementations have subtle differences that make it hard to eliminate inconsistencies and edge cases—and it's hard to discover those in the first place because Bash is all but untestable. - It can leave your system in an awkward "in-between" state, with some installation actions successful and others not. The installer may, for example, successfully create a root volume but then fail to create the appropriate system users. But awkward or broken state is something that Nix is _supposed to free you from_. - Relatedly, it offers no built-in way to uninstall Nix, leaving people to seek out help on [Github](https://github.com/NixOS/nix/issues/1402#issuecomment-312496360) and [Stack Overflow](https://stackoverflow.com/questions/51929461/how-to-uninstall-nix). But we suspect it would help to spur Nix adoption if users could install Nix with complete confidence that they could undo all system changes required by the installation process with a single command: `/nix/nix-installer uninstall` (the installer removes itself when you uninstall). We didn't think that "improve the existing Bash script" was a sufficiently radical approach, so we began discussing a vision for a new installer internally. Later, we saw the [installer discussion](https://discourse.nixos.org/t/nix-installer-workgroup/21495) on the [NixOS Discourse](https://discourse.nixos.org) and enthusiastically joined the installer working group. In that group, we've kept group members apprised of our progress and discussed what the Determinate Nix Installer would need to provide to be broadly adoptable by the Nix community. We think it's well on the way to being that, but we hope that this release prompts many rounds of community feedback. ## Core differentiators We believe that the Determinate Nix Installer offers two main advances: - We wrote it in **Rust** instead of Bash. The chief advantage of Rust here is that it deftly sidesteps any conceivable problems with differing Bash implementations, and because Rust supports a wide range of build targets, the Determinate Nix Installer can support a wide range of systems, including systems without Bash installed. Beyond that, Rust offers robust error handling, a fantastic type system, and great libraries like [clap](https://github.com/clap-rs/clap) for building CLIs and [Tokio](https://tokio.rs) for asynchronous programming. The installer was and will remain a joy to develop, something that can rarely be said of Bash scripts. - We introduced the concept of an installation **receipt**. This is a JSON file that indicates what has already been accomplished by the installer—and what hasn't. It's stored under `/nix` and keeps all the information the installer needs to _uninstall_ Nix seamlessly. That means deleting all of the volumes, directories, files, users, and groups that the installer created as well as surgically undoing changes to files you need to keep around (like `bashrc`). ## A flexible installer Above all, we want our installer to work seamlessly in any environment where Nix can work its magic: - On your desktop, laptop, or server, whether manually or using automated approaches like mobile device management (MDM). - In continuous integration and continuous deployment environments. We provide a [GitHub Action](https://github.com/DeterminateSystems/determinate-nix-action) for the Determinate Nix Installer, but it should be adaptable to any CI system. - In virtual machines and [OCI](https://opencontainers.org) containers. To that end, we've provided an opinionated default experience configurable enough to meet more specific requirements. The opinionated part is that the Determinate Nix Installer installs Nix with [Nix flakes](https://zero-to-nix.com/concepts/flakes) and the [unified CLI](https://zero-to-nix.com/concepts/nix#unified-cli) enabled. Although flakes and the unified CLI remain experimental features of Nix, we're confident that new users should adopt them now and that more seasoned Nix users should [make the transition ASAP](https://zero-to-nix.com/concepts/flakes). In terms of configurability, the Determinate Nix Installer provides a range of CLI flags that you can use to customize the experience. To give some examples, running nix-installer install --no-confirm disables the default manual confirmation step (good for things like setup scripts), `nix-installer install --init none` installs Nix without needing an init system like [systemd](https://systemd.io/) (good for things like Docker containers), and `nix-installer install --extra-config` enables you to supply extra Nix configuration to be written directly to your [`nix.conf`](https://nixos.org/manual/nix/stable/command-ref/conf-file.html) file. Over time, we'll likely add more configuration options in response to community feedback. Feel free to file an [issue](https://github.com/DeterminateSystems/nix-installer/issues) if you run into problems. ## The future of the installer As a company, we're open to a wide variety of possible trajectories for the Determinate Nix Installer. We've [licensed it](https://github.com/DeterminateSystems/nix-installer/blob/main/LICENSE) under the highly permissive [GNU Lesser General Public License v2.1](https://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html)—under which Nix itself is licensed—which should provide little obstruction to eventually making it the official installer if that's what the community desires. We've also released it [as a Rust library](https://docs.rs/nix-installer/0.3.0/nix_installer/) that anyone can make use of, including a potential reimagined official installer that borrows logic from the Determinate Nix Installer but provides a different set of defaults. The initial feedback for the Determinate Nix Installer has been uniformly positive, and we're thrilled to finally release it to the community after months of hard work. We know that it needs to prove itself in a wide variety of situations and that it will need several rounds of feedback and bug fixes. We're eager to undergo that process in conjunction with the Nix community. We take the core tenets of Nix seriously. True to our name, we want to make software systems as **_determinate_** as they can conceivably be. Making Nix installation robust, seamless, and trivially reversible is a major step in that direction. --- **Introducing Zero To Nix** Published: January 23, 2023 URL: https://determinate.systems/blog/zero-to-nix We at [Determinate Systems](https://determinate.systems) are pleased to announce our initial release of [Zero to Nix](https://zero-to-nix.com/), a brand new learning resource for [Nix](https://nixos.org). You can check it out at [zero-to-nix.com](https://zero-to-nix.com). Zero to Nix is mostly targeted at two groups: - People who are curious about Nix but haven't yet taken the time to really explore it (perhaps because they've felt intimidated by it). - People who are pretty sure they would benefit from Nix and have _tried_ to cross the chasm to using it in their daily workflows but haven't gotten there yet. We're confident that both of these groups stand to benefit from Zero to Nix. It presents information in a streamlined, narrative format that's geared toward generating fresh interest in Nix and unblocking those who have felt stymied in the past. ## Information architecture Upon initial launch, Zero to Nix is divided into two basic content domains: the quick start and the concept docs. ### [Quick start](https://zero-to-nix.com/start) This takes you all the way from not even having Nix on your machine to: - [Installing](https://zero-to-nix.com/start/install) Nix using [Nix Installer](https://github.com/determinateSystems/nix-installer), a next-generation tool from us at Determinate Systems. - [Running](https://zero-to-nix.com/start/nix-run) programs on the command line using Nix. - Creating [development environments](https://zero-to-nix.com/start/nix-develop) using Nix. - [Building](https://zero-to-nix.com/start/nix-build-nixpkgs) packages using Nix. - [Searching](https://zero-to-nix.com/start/nix-search) for packages in Nixpkgs and elsewhere in the Nix ecosystem. And much more. The quick start is geared toward generating those initial "a-ha!" moments that make people feel empowered to go further. ### [Concept docs](https://zero-to-nix.com/concepts) While there's already a lot of conceptual documentation about Nix out there, much of it is tricky to navigate and useful only to seasoned pros. Zero to Nix's conceptual docs are intended _not_ to be comprehensive or the final word. On the contrary, they're meant to both provide digestible nuggets of content that smooth the landing into Nix and to direct readers to docs that _are_ meant to be more comprehensive. ## Why a new learning resource? The Nix ecosystem already has a _lot_ of documentation in it. There's the [Nixpkgs manual](https://nixos.org/manual/nixpkgs/stable/), the [NixOS manual](https://nixos.org/manual/nixos/stable/), [nix.dev](https://nix.dev/), the [Nixpkgs](https://github.com/nixOS/nixpkgs) codebase, [Nix Pills](https://nixos.org/guides/nix-pills/), and more. Zero to Nix is _not_ intended as a replacement for any of those sources and we will certainly not dissuade you from consulting them (in fact, some of us at Determinate Systems have contributed a great deal to them). But these sources have one thing in common: they don't really excel at onboarding new users. The decision to create a new resource from scratch isn't one that we took lightly. Our reasoning in a nutshell: - Although Nix flakes remain experimental, we believe that they are the future of Nix. Anyone beginning their Nix journey today should center their learning and experimentation around Nix flakes and the new [unified `nix` CLI](https://zero-to-nix.com/concepts/nix#unified-cli) rather than Nix channels and the `nix-*` CLI tools (`nix-env`, `nix-shell`, `nix-build`, etc.). And many new Nix users are doing precisely that. But because both flakes and the new CLI are experimental, official documentation sources are barred from featuring them. And with good reason! Official sources need to be careful and conservative with experimental features; we wanted to proceed differently. - We like the [official Nix installation script](https://nixos.org/download) well enough, and all of us at Determinate Systems have used it numerous times. But it has some shortcomings that make it less than ideal. First, it doesn't provide a solution to the tricky problem of uninstalling Nix, requiring you to manually undo the changes that the script makes to your system. Second, it requires you to manually enable flakes and the [unified CLI](https://zero-to-nix.com/concepts/nix#unified-cli). Nix Installer, on the other hand, seamlessly undoes all of the changes it makes to your system and enables flakes and the unified CLI by default. We'll be saying much more about Nix Installer soon. - We wanted to be bold and move quickly. When you're working within the confines of official projects, you need to work carefully, building consensus, having often-difficult conversations, and striving for small wins and incremental progress. With Zero to Nix, we wanted to inject a fresh approach and perspective into the mix and our cumulative experience in the Nix community suggested that going outside official channels was the best way forward. - We wanted full control over the site generation process and the freedom to include interactive elements like widgets for selecting your preferred programming language and a dark mode option. The official Nix documentation sources use more "vanilla" static site generators. In case you're curious, Zero to Nix is built using the lovely [îles](https://iles.pages.dev/) framework, with [Vue](https://vuejs.org/) for its templating logic and [Tailwind](https://tailwindcss.com/) for styling and [Nano Stores](https://github.com/nanostores/nanostores) for the limited amount of state management the site needs. ## How to contribute There are many ways that you can help out with Zero to Nix: - [Pull requests](https://github.com/DeterminateSystems/zero-to-nix/pulls) welcome! Zero to Nix is licensed under [Creative Commons](https://github.com/DeterminateSystems/zero-to-nix/blob/main/LICENSE) and we welcome pull requests for both written content and aesthetics/UX. - File [issues](https://github.com/DeterminateSystems/zero-to-nix/issues). As with pull requests, we warmly welcome issues related to the content of the site as well as aesthetics and UX. We recommend using issues to identify problems and discussions (see directly below) for more open-ended purposes. - Start a [discussion](https://github.com/DeterminateSystems/zero-to-nix/discussions). Do you want to see a new form of content on Zero to Nix? Do you have questions about Nix or our Nix Installer? This is a great place to interact with us at Determinate Systems and others in the community. ## Shortcomings In its current state, we think that Zero to Nix will be of immediate benefit to many, from general technology enthusiasts to decision makers considering Nix to people who have tried to learn Nix before but gave up for whatever reason. But providing a full-fledged "front end" to Nix is quite a task and we're not quite where we want to be. Most importantly, the [concept docs](https://zero-to-nix.com/concepts) will be a major area of focus in the coming weeks. We currently have docs for most of the concepts we'd like to cover in Zero to Nix but some of those docs are a bit thin. We're more confident in the quick start as we've performed several user studies for that content (huge thanks to those who participated!). But even with the quick start, the community will be the ultimate judge. People will point out missing content, readers will get confused by the way we present certain concepts, people will have unexpected problems using Nix on some platforms. C'est la vie. We're prepared to do whatever it takes to turn Nix from impenetrable to an indispensable part of people's workflows. ## Our goals for Zero to Nix Our overarching goal at [Determinate Systems](https://determinate.systems/) is to enable people to more fully harness [Nix](https://nixos.org)'s potential. The good news is that Nix is already extremely powerful, providing a wide range of features that can immensely benefit teams and entire organizations: declarative development environments and Linux systems, fully reproducible package builds, and much more. But we know that Nix can be a tough nut to crack, and the existing documentation sources often come up short for Nix users of _all_ knowledge levels. Many of us in the Nix community feel that Nix is about to reach a major inflection point—if it hasn't already—and we see Zero to Nix as a resource that can help Nix cross the chasm. --- **Nix on the Steam Deck** Published: December 21, 2022 URL: https://determinate.systems/blog/nix-on-the-steam-deck When I first started using Linux in 2006 I remember dreaming of a Linux Console. The idea maybe wasn't so far fetched at the time, the PlayStation 3 had just been released with OtherOS support which allowed users to install Linux (or BSD). Still, it seemed that a Linux-first console would only ever be a dream. Now in 2022, Valve's Steam Deck is a hackable Linux-first portable console. Today, we'll be putting Nix on it, because what's Linux without Nix? > Just wanna try it? [Jump to the fun part.](#enabling-an-install) Want NixOS > instead? > > A different, spicier kind of fun can be found > [here](https://github.com/Jovian-Experiments/Jovian-NixOS). The Steam Deck is a portable computer that has a Nintendo Switch-like form factor and touts an AMD x86_64 processor. It has WiFi, Bluetooth, and a USB-C port which you can plug a hub into, allowing the attachment of HDMI, mice, keyboards, power, or ethernet cables. It runs a flavour of Arch Linux called SteamOS, and guides users to use [Flatpak](https://github.com/flatpak/flatpak) and [Flathub](https://flathub.org/home). This is a fantastic solution and provides users access to a wide variety of software, but as a developer I tend to want more exotic stuff that exists in [`nixpkgs`](https://github.com/NixOS/nixpkgs). In case you'd not seen one yet, here's a picture of mine: My Deck, alongside the peripherals I use with it: PS5 Controller, a USB-C hub, and an Ergodox. Installing Nix on the Steam Deck has a few special steps. Let's review how a Nix install process looks, then how the Deck works, and finally we can explore a working approach to install Nix. ## How a Nix install works A normal Nix install process on Linux works roughly like this: - Create a folder called `/nix` - Unpack the Nix distribution tarball into `/nix` - Create some Nix daemon users and a group (affecting `/etc`) - Call `systemctl link` on some systemd units from `/nix` (affecting `/etc`) - Sprinkle some magic in the detected shell profiles (in `/etc`) to ensure `nix` is on `$PATH` On Mac, where creating a `/nix` is forbidden (by the creators, Apple), we can modify [`/etc/synthetic.conf`](https://keith.github.io/xcode-man-pages/synthetic.conf.5.html) to create a stub which we can mount an APFS volume to. The installation otherwise proceeds as normal. On the Steam Deck, creating `/nix` also requires special steps. Unfortunately, there is no feature similar to `/etc/synthetic.conf`. Why does the Deck need these special steps? Let's take a look at the Steam Deck itself and figure out why we can't just run the familiar Nix installer. ## The Deck & SteamOS The Steam Deck ships with an [Arch Linux](https://archlinux.org/) based distribution called [SteamOS](https://store.steampowered.com/steamos) -- a [special image](https://help.steampowered.com/en/faqs/view/1b71-edf2-eb6d-2bb3) of it to be even more precise. Normally Arch Linux is a perfectly fine target for Nix, but there are a couple particularities around this distribution that impact how we can install Nix. ### Disk Topology The Deck uses an A/B boot system [(like some Android phones)](https://source.android.com/docs/core/ota/ab), which means it has two parallel installations, booting into one and updating the other. This means, if it ever fails after an update it can safely roll back to a known good state. > **NixOS user?** Sound familiar? It's like the generation selector in your bootloader, but instead of pointing your boot to different Nix store paths, it points to entirely different partitions! ``` (deck@steamdeck ~)$ sudo gdisk /dev/nvme0n1 -l Number Start (sector) End (sector) Size Code Name 1 2048 133119 64.0 MiB EF00 esp 2 133120 198655 32.0 MiB 0700 efi-A 3 198656 264191 32.0 MiB 0700 efi-B 4 264192 10749951 5.0 GiB 8304 rootfs-A 5 10749952 21235711 5.0 GiB 8304 rootfs-B 6 21235712 21759999 256.0 MiB 8310 var-A 7 21760000 22284287 256.0 MiB 8310 var-B 8 22284288 1000215175 466.3 GiB 8302 home ``` See how there is `A` and `B` copies of most partitions? This looks a heck of a lot different than my development machine: ``` Number Start (sector) End (sector) Size Code Name 1 2048 2099199 1024.0 MiB EF00 efi 2 2099200 3907029134 1.8 TiB 8309 encrypt ``` Checking for encryption with `blkid | grep crypto_LUKS` showed all partitions were unencrypted, this makes sense since the Deck never asks for a password, even for `sudo`, until you set one. It's a bit unfortunate Valve did not opt to protect their user's data in the event this portable device was stolen, but it's room to improve. This A/B boot system means even if `rootfs` partitions get modified, those changes may get wiped out at any time. The system may update or choose to boot into the other 'letter' for some other reason. We want something that is update-proof and will survive a change of 'letter'. One partition that persists across reboots and has enough space to contain a thick, chunky Nix store is the `home` partition. Our Nix install can keep persistent data there. ### Read-Only Filesystem Reviewing the `mount` output is a bit misleading. While the `/` mount says it is `rw`, it is normally not. ``` (deck@steamdeck ~)$ mount | grep /dev/nvme /dev/nvme0n1p4 on / type btrfs (rw,relatime,ssd,space_cache=v2,subvolid=5,subvol=/) /dev/nvme0n1p6 on /var type ext4 (rw,relatime) /dev/nvme0n1p8 on /home type ext4 (rw,relatime,x-systemd.growfs) /dev/nvme0n1p8 on /opt type ext4 (rw,relatime) /dev/nvme0n1p8 on /root type ext4 (rw,relatime) /dev/nvme0n1p8 on /srv type ext4 (rw,relatime) /dev/nvme0n1p8 on /var/cache/pacman type ext4 (rw,relatime) /dev/nvme0n1p8 on /var/lib/docker type ext4 (rw,relatime) /dev/nvme0n1p8 on /var/lib/flatpak type ext4 (rw,relatime) /dev/nvme0n1p8 on /var/lib/systemd/coredump type ext4 (rw,relatime) /dev/nvme0n1p8 on /var/log type ext4 (rw,relatime) /dev/nvme0n1p8 on /var/tmp type ext4 (rw,relatime) (deck@steamdeck ~)$ sudo touch /boop touch: cannot touch '/boop': Read-only file system ``` This isn't a scary vendor lockdown security feature or anything, it's mostly to prevent the user from being surprised when the A/B boot happens. SteamOS comes with a `steamos-readonly` executable we can use to toggle this read-only feature at any time, this can allow us to make small changes to the root filesystem as long as we don't expect them to persist across boots. Because of this, if we wanted, we could create a `/nix` path on the `rootfs` each boot by making the root momentarily writable. Not all of the device is read-only though! We can write to places like `/etc/`, but not to `/lib`, `/usr`, or `/bin`. ``` (deck@steamdeck ~)$ sudo touch /etc/boop (deck@steamdeck ~)$ sudo rm /etc/boop (1)(deck@steamdeck ~)$ sudo touch /lib/boop touch: cannot touch '/lib/boop': Read-only file system (1)(deck@steamdeck ~)$ sudo touch /bin/boop touch: cannot touch '/bin/boop': Read-only file system (1)(deck@steamdeck ~)$ sudo touch /usr/boop touch: cannot touch '/usr/boop': Read-only file system ``` Recalling the rough steps from the install process, this isn't a problem! So long as we work out the machinery to ensure `/nix` is available, the Steam Deck looks otherwise like a normal system to Nix. ## Enabling an Install As we discovered, creating the `/nix` directory in a safe way that persists will be our primary challenge. Since it wouldn't be a great idea to store the Nix Store on the `rootfs` partitions, we must decide somewhere else. The most immediately obvious answer is `/home/nix`, since that is a large, persistent location. With an existing `/home/nix`, we can use a [bind mount](https://www.baeldung.com/linux/bind-mounts) to mount that to `/nix`. First, a `/nix` path needs be created somehow! Luckily, with `/etc` writable, we can drop systemd units into [`/etc/systemd/system`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html) that will set up `/nix` for us. We'll create a `nix-directory.service` unit which creates the `/nix` path, and a `nix.mount` unit which depends on that. Sadly, that's not quite enough to enable a full install though. Since the Nix install process involves `systemctl link $UNIT`, some of the systemd units are not available during systemd's startup. Therefore we must reload the systemd daemon itself after the `nix.mount` unit is started. In order to do that, we follow the same method as [Flatcar Linux](https://www.flatcar.org/) does [here](https://github.com/flatcar/init/blob/flatcar-master/systemd/system/ensure-sysext.service). Let's cover what these units look like then test them out with the Nix installer! If you're feeling brave [I invite you](#an-invitation-to-experiment) to help us test an experimental Nix installer we've been working on which has a special codepath just for the Steam Deck. Otherwise, follow along below to try the traditional install script. But first, just in case: - **Not sure how to get to 'Desktop mode'?** Hit the Steam button, go to 'Power', go to 'Switch to Desktop' - **Not sure how to get a terminal?** In 'Desktop Mode' hit the logo in the bottom left corner, in the search bar type "Terminal", select 'Konsole' - **Not sure how to edit files?** You can use `vim` if you are familiar, otherwise try `nano` from the terminal. ### Putting it all together > **Want to follow along without a Deck?** Learn how to set up a Deck VM with [this article](https://blogs.igalia.com/berto/2022/07/05/running-the-steam-decks-os-in-a-virtual-machine-using-qemu/). There are only four Steam Deck specific steps, three are to create the systemd units. The final one is to enable one of those units. Create the systemd units at the noted paths, I suggest using a keyboard plugged into the Deck if you can, or enable SSH via `sudo systemctl start sshd`, reviewing the IP address via `ip a`, and setting a password. If those options are unavailable, hit the [**Steam and X**](https://help.steampowered.com/en/faqs/view/671A-4453-E8D2-323C) buttons to summon the keyboard. ```ini showLineNumbers # /etc/systemd/system/nix-directory.service [Unit] Description=Create a `/nix` directory to be used for bind mounting PropagatesStopTo=nix-daemon.service PropagatesStopTo=nix.mount DefaultDependencies=no [Service] Type=oneshot ExecStart=steamos-readonly disable ExecStart=mkdir -vp /nix ExecStart=chmod -v 0755 /nix ExecStart=chown -v root /nix ExecStart=chgrp -v root /nix ExecStart=steamos-readonly enable ExecStop=steamos-readonly disable ExecStop=rmdir /nix ExecStop=steamos-readonly enable RemainAfterExit=true ``` The above unit is the first in our chain of units, it checks if a `/nix` folder exists, and if necessary, calls `steamos-readonly disable`, creates `/nix`, then calls `steamos-readonly enable` again. It also attempts to do some cleanup as it stops, but that part is unnecessary. ```ini showLineNumbers # /etc/systemd/system/nix.mount [Unit] Description=Mount `/home/nix` on `/nix` PropagatesStopTo=nix-daemon.service PropagatesStopTo=nix-directory.service After=nix-directory.service Requires=nix-directory.service ConditionPathIsDirectory=/nix DefaultDependencies=no RequiredBy=nix-daemon.service RequiredBy=nix-daemon.socket [Mount] What=/home/nix Where=/nix Type=none DirectoryMode=0755 Options=bind ``` This mount unit performs a bind mount from `/home/nix` to `/nix`. It'll create `/home/nix` for us, but sadly it cannot create `/nix`, relying on the `nix-directory.service` before it. ```ini showLineNumbers # /etc/systemd/system/ensure-symlinked-units-resolve.service [Unit] Description=Ensure Nix related units which are symlinked resolve After=nix.mount Requires=nix-directory.service Requires=nix.mount DefaultDependencies=no [Service] Type=oneshot RemainAfterExit=yes ExecStart=/usr/bin/systemctl daemon-reload ExecStart=/usr/bin/systemctl restart --no-block nix-daemon.socket [Install] WantedBy=sysinit.target ``` This final unit in the chain restarts the systemd daemon, allowing it to properly resolve any previously broken symlinks during the boot, before starting or enabling them if necessary. > **Tailscale user?** A similar strategy can be used after performing `systemd-sysext merge` if you happen to also use [Tailscale on your Steam Deck](https://tailscale.com/blog/steam-deck/) to make sure it starts at boot. After creating the units, we need to enable (and start) the last, causing the ones it requires to also start: ```bash sudo systemctl enable --now ensure-symlinked-units-resolve.service ``` Now we can just run the [Nix installer](https://nixos.org/manual/nix/stable/quick-start.html) like normal: ```bash sh <(curl -L https://nixos.org/nix/install) --daemon ``` Follow the prompts, call `exec $SHELL` (or open a new shell, or reboot) and Nix should work on command line! ``` (deck@steamdeck ~)$ nix-instantiate --eval -E '1 + 1' 2 (deck@steamdeck ~)$ nix-build '' -A hello these 4 paths will be fetched (6.73 MiB download, 31.05 MiB unpacked): /nix/store/34xlpp3j3vy7ksn09zh44f1c04w77khf-libunistring-1.0 /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163 /nix/store/5mh5019jigj0k14rdnjam1xwk5avn1id-libidn2-2.3.2 /nix/store/g2m8kfw7kpgpph05v2fxcx4d5an09hl3-hello-2.12.1 copying path '/nix/store/34xlpp3j3vy7ksn09zh44f1c04w77khf-libunistring-1.0' from 'https://cache.nixos.org'... copying path '/nix/store/5mh5019jigj0k14rdnjam1xwk5avn1id-libidn2-2.3.2' from 'https://cache.nixos.org'... copying path '/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163' from 'https://cache.nixos.org'... copying path '/nix/store/g2m8kfw7kpgpph05v2fxcx4d5an09hl3-hello-2.12.1' from 'https://cache.nixos.org'... /nix/store/g2m8kfw7kpgpph05v2fxcx4d5an09hl3-hello-2.12.1 (deck@steamdeck ~)$ ./result/bin/hello Hello, world! ``` Feel free to reboot a few times, or even update your Steam Deck. As far as I've experimented, it should keep working! If you cause a instant, hard, full power loss (such as Ctrl+C'ing the VM) before it can properly `fsync()`, you may see an error like `error: expected string 'Derive(['`. To resolve this error, run `nix store gc`. You can avoid this by running `sync` before killing the device. ### An invitation to experiment Part of the reason we wanted to explore Nix on the Steam Deck is that we're currently experimenting with a new Nix installer, and we were curious what we could learn from adding support for a specific device which had special requirements, such as the Steam Deck. If you feel like experimenting **(and don't mind things breaking)** feel encouraged to try out our prototype! Don't worry, if you don't like it, it includes an uninstaller so you can roll back and do your install with the traditional scripts. You can run it like this: ```bash curl -L https://install.determinate.systems/nix | sh -s -- install steam-deck ``` If you don't feel like being experimental, this is what it looks like: ``` (deck@steamdeck ~)$ curl -L https://install.determinate.systems/nix | sh -s -- install steam-deck % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 15739 100 15739 0 0 22981 0 --:--:-- --:--:-- --:--:-- 22981 info: downloading installer https://install.determinate.systems/nix/nix-installer-x86_64-linux ./nix-installer install steam-deck `nix-installer` needs to run as `root`, attempting to escalate now via `sudo`... Nix install plan (v0.0.0-unreleased) Planner: steam-deck Planner settings: * persistence: "/home/nix" * channels: ["nixpkgs=https://nixos.org/channels/nixpkgs-unstable"] * nix_build_user_id_base: 3000 * extra_conf: [] * modify_profile: true * force: false * daemon_user_count: 32 * nix_build_group_name: "nixbld" * nix_build_user_prefix: "nixbld" * nix_package_url: "https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz" * nix_build_group_id: 3000 These actions will be taken (`--explain` for more context): * Create directory `/home/nix` * Create or overwrite file `/etc/systemd/system/nix-directory.service` * Create or overwrite file `/etc/systemd/system/nix.mount` * Create or overwrite file `/etc/systemd/system/ensure-symlinked-units-resolve.service` * Enable (and start) the systemd unit ensure-symlinked-units-resolve.service * Fetch `https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz` to `/nix/temp-install-dir` * Create build users (UID 3000-3032) and group (GID 3000) * Create a directory tree in `/nix` * Move the downloaded Nix into `/nix` * Setup the default Nix profile * Configure Nix daemon related settings with systemd * Place the Nix configuration in `/etc/nix/nix.conf` * Place channel configuration at `/root/.nix-channels` * Configure the shell profiles * Enable (and start) the systemd unit nix-daemon.socket Proceed? (y/N): y INFO Step: Create directory `/home/nix` INFO Step: Create or overwrite file `/etc/systemd/system/nix-directory.service` INFO Step: Create or overwrite file `/etc/systemd/system/nix.mount` INFO Step: Create or overwrite file `/etc/systemd/system/ensure-symlinked-units-resolve.service` INFO Step: Enable (and start) the systemd unit ensure-symlinked-units-resolve.service INFO Step: Provision Nix INFO Step: Configure Nix INFO Step: Enable (and start) the systemd unit nix-daemon.socket (deck@steamdeck ~)$ . /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh (deck@steamdeck ~)$ nix run nixpkgs#hello Hello, world! ``` Hate it? Uninstall it: ``` (deck@steamdeck ~)$ /nix/nix-installer uninstall ``` Our prototype has the working name of `nix-installer`. It supports different installation 'planners' (such as the `steam-deck`), can be used as a Rust library, has fine grained logging, and can uninstall a Nix it installed. It has no runtime dependencies (though it will try to `sudo` itself if you forget) or build time dependencies (other than Rust/C compilers) and should build trivially inside or outside Nix for x86_64 and aarch64, Linux (`glibc` or `musl` based) and Mac. We are currently distributing fully reproducible and hermetic `nix` based **experimental** builds for all supported platforms. The installer is Open Source (LGPL) and written in entirely in Rust. (Nix is still not in Rust -- sorry!) You are welcome to explore the code [here](https://github.com/DeterminateSystems/nix-installer). Don't worry, we're excited to talk about it at length in a future article. Stay tuned for more! > We've been working with other installer working group contributors like (alphabetical) [Cole](https://github.com/colemickens), [Michael](https://github.com/mkenigs), [Solène](https://dataswamp.org/~solene/index.html), [Théophane](https://github.com/thufschmitt), [Travis](https://t-ravis.com/), and others to build `nix-installer` and better understand what a next-generation Nix installer would look like, thank you so much for all your help, hard work, and advice. ## Conclusion We explored how the Steam Deck takes certain measures to protect users from accidentally losing changes when the system updates and swaps due to its A/B booting, we also explored how we can use persistent systemd units to create a `/nix` path on the Steam Deck which bind mounts to a persistent `/home/nix` directory. In order to ensure that the units linked from the `/nix` path are loaded, we also learnt we can have a unit which reloads the systemd daemon, and how this resolves the issue. Using these techniques, we successfully installed Nix on the Steam Deck using both the traditional installer, as well as a prototype that we've been working on. --- **Deploying Nix-built containers to Kubernetes** Published: November 21, 2022 URL: https://determinate.systems/blog/nix-to-kubernetes While it's great to fall in love with [Nix](https://nixos.org) and want to use it to build and configure just about everything, let's face it: few of us are in a position to do so, especially in the workplace. "Hey team, let's Nixify our _entire_ stack!" is unlikely to endear you to your coworkers at daily standup. Fortunately, Nix is good at enough things that you can reap many of its potential benefits by incorporating it into only a subset of an already existing software pipeline. In this post, I'll provide a concrete example of this by using Nix inside a fairly standard [DevOps](https://en.wikipedia.org/wiki/DevOps) pipeline that builds [Docker](https://docker.com) images and deploys them to [Kubernetes](https://kubernetes.io) in CI. Nix does a lot of heavy lifting in this scenario but it doesn't do everything: it doesn't stand up any infrastructure and it doesn't handle deployment. That's all left to popular, trusted tools and platforms from outside the Nix universe. I hope that this example inspires you to find ways to incrementally introduce Nix into your own stacks. ## The deployment scenario In my example scenario, which you can find in the [nix-to-kubernetes](https://github.com/DeterminateSystems/nix-to-kubernetes/) repo under the [Determinate Systems](https://github.com/DeterminateSystems/nix-to-kubernetes) organization, I use Nix for two things: - To create cross-platform [development environments](/blog/nix-direnv) for both local machines and the [continuous integration level](/blog/nix-github-actions) (which runs on GitHub Actions). - To build a [Go](https://golang.org/) web service into [Docker](https://docker.com/) images that can be deployed on Kubernetes (or most other container orchestration platforms). - All of that is configured in the repo's [flake](https://github.com/DeterminateSystems/nix-to-kubernetes/blob/main/flake.nix), which amounts to a few dozen lines. What Nix doesn't do here: - Stand up any infrastructure. Instead, [Terraform](https://terraform.io/) uses an [HCL](https://github.com/DeterminateSystems/nix-to-kubernetes/blob/main/main.tf) configuration to handle that. - Deploy the container. Instead, [kubectl](https://kubernetes.io/docs/reference/kubectl/) deploys the container to Kubernetes, which is configured using a straightforward [YAML file](https://github.com/DeterminateSystems/nix-to-kubernetes/blob/main/kubernetes/deployment.yaml). ## The benefits of Nix What would this scenario look like without Nix? I suspect it would usually look something like this: - Contributors would be on their own to create their development environment. The README would perhaps list necessary dependencies—[Go](https://golang.org/), [Terraform](https://terraform.io/), [kubectl](https://kubernetes.io/docs/reference/kubectl/), and others—but there'd be no guarantee that the "works on my machine" problem would be overcome the way it can [with Nix](/blog/nix-direnv). - The CI pipeline would use third-party Actions, which are problematic for reasons I lay out [here](/blog/nix-github-actions). - The Docker CLI and a [Dockerfile](https://docs.docker.com/engine/reference/builder/) would be used to build Docker images. Dockerfiles are a "good enough" construct for defining container build logic, but they're not reproducible by default (the same `apt-get install` command can yield different results on different days, for example) and their procedural nature—`COPY` this, `ADD` that, `RUN` this—is a source of great consternation for many. Contrast that approach with [this highly concise build logic](https://github.com/DeterminateSystems/nix-to-kubernetes/blob/75945c01fb214feb2cc0e023c9b5bb5e699e09fb/flake.nix#L49-L58) in Nix. By contrast, using Nix here—a relatively straightforward Nix [flake](https://github.com/DeterminateSystems/nix-to-kubernetes/blob/main/flake.nix)—provides a standardized, cross-platform dev environment that's synced with the CI environment as well as a fully declarative Docker image build, a major improvement over the known brittleness of procedural Dockerfiles. ## Implications This project shows that Nix can not just unobtrusively but quite comfortably live alongside tools and processes that have zero awareness of Nix. I certainly could have "Nixified" other parts of this pipeline but deliberately held back. Introducing Nix into an organization can be a daunting process, especially if you want to boil the proverbial ocean and convert everything in sight—developing, building, deploying—to Nix. But what I hope to have shown here is that you can introduce Nix in a gentle, piecemeal way that yields immediate benefits while introducing few if any drawbacks. Using Nix to run a service locally and package it as a Docker image is a low-risk way to introduce Nix into a multifaceted software pipeline. So go head, pass this post along to your teammates or, even better, CTO or director of engineering. If you—or your superiors—have any lingering questions, feel more than free to [reach out](mailto:hello@determinate.systems) to us. --- **Streamline your GitHub Actions dependencies using Nix** Published: October 31, 2022 URL: https://determinate.systems/blog/nix-github-actions Let me say at the outset: I enjoy using [GitHub Actions](https://github.com/features/actions) as a continuous integration system. It has a nice UI, I'm especially fond of the [matrix variations](https://prettier.io/) feature, and it's nice to get started—after all, my code is already "there" on GitHub. One thing that I do _not_ love about the platform: Actions as [third-party dependencies](https://github.com/marketplace?type=actions). In this post, I'll argue that Actions are a problematic and often superfluous abstraction and that you should consider using [Nix](https://nixos.org/) to make your pipelines dramatically more reproducible and ergonomically sound. ## The problem with Actions While there are important exceptions, Actions are typically little more than wrappers around a single executable and yet require substantial boilerplate to work properly. Take [`creyD/prettier_action`](https://github.com/marketplace/actions/prettier-action) as an example. This Action installs [Prettier](https://prettier.io/) in your CI environment and enables you to specify the commands you want to run with it. Here's an example configuration: ```yaml showLineNumbers - name: Prettify repo code uses: creyD/prettier_action@v4.2 with: prettier_options: --write **/*.{js,jsx,ts,tsx} only_changed: true ``` This is short enough to set up—just a few lines of YAML—but have a look at the [repo](https://github.com/creyD/prettier_action) that defines this action. You'll see an [`action.yml`](https://github.com/creyD/prettier_action) file that defines which with options are available and how the action is [run](https://github.com/creyD/prettier_action/blob/master/action.yml#L58-L78). The actual logic of the action is then defined in a [shell script](https://github.com/creyD/prettier_action/blob/master/action.yml#L58-L78). This is pretty substantial song-and-dance for what would be a command invocation if Prettier were already installed in the environment. To be clear, there's nothing _wrong_ with `prettier_action` per-se and I don't intend to single it out. I chose it solely because it's representative. So why all this boilerplate? It's basically the cost you pay for light-weight installation. Need a dependency in your pipeline in just a few lines of code? Boom, you got it. No need to fuss with [Homebrew](https://brew.sh) or [yum]() or [apt]() or anything else; the creators of the Action have handled that tangled business for you (hopefully). But this approach harbors some significant drawbacks: - The specific Action you want **may not exist**, in which case you'll need to find a way to get the desired tool(s) into your CI environment or create an Action yourself (with all the boilerplate that involves). - **Anxiety of choice**. There may be five different Actions that do more or less the same thing, leaving you with a vetting problem you'd likely prefer to avoid. - **Platform support**. The Action may install the tool you want on Linux but not macOS, for example, so if your [runner](https://docs.github.com/en/actions/using-jobs/choosing-the-runner-for-a-job) or [matrix strategy](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs) includes a `macos-*` machine, that Action might be full-on broken for some of your jobs. And worst of all, even if you _do_ find an Action that's Just Right™️, you have two remaining problems: 1. The **time** it took you to ascertain that the Action meets your needs—digging through various docs and files and engaging in `git push` shenanigans—is a steep price to pay just to play by another platform's rules. 2. You can't _really_ run that Action **locally**. Tools like [act](https://github.com/nektos/act) do their best to make this possible, but my experiments with act have shown it to be a rough half-solution. Ideally, you'd be able to run most or all of your CI pipeline locally, but if you use third-party Actions you're setting yourself up for frustrating discrepancies between your CI environment and what you and your team can run locally. ## The Nix alternative As promised, I'm going to present [Nix](https://nixos.org/) as a clear alternative to using third-party Actions in your GitHub CI pipelines. The key Nix feature I want to showcase here is [Nix shell environments](https://nixos.wiki/wiki/Development_environment_with_nix-shell). In a nutshell, you can use Nix expressions to declare which dependencies you want to make available inside an isolated shell environment for your project. Here's an example of a shell environment with [Go](https://golang.org) 1.18, [Prettier](https://prettier.io), [Cargo](https://doc.rust-lang.org/cargo/), [Python](https://python.org) 3.8, and [OpenSSL](https://openssl.org) installed: ```nix showLineNumbers { devShells.default = pkgs.mkShell { buildInputs = with pkgs; [ go_1_18 nodePackages.prettier cargo python38 openssl ]; }; } ``` Nix shell environments have the virtue of being highly replicable across platforms, which means that they're an ideal solution to the "works on my machine" problem for your CI environment. You may not _always_ be able to easily install every tool on every system—some things may not be available on macOS via Nix, for example—and that's something to always be on the lookout for. When you define a shell environment using Nix (with [flakes](https://www.notion.so/Streamlining-your-GitHub-Actions-dependencies-using-Nix-5dcf0dc66b884d7fa82efe9f696562cb) enabled), you can enter the default local environment (as in the example above) by running `nix develop` or a more specific environment using `nix develop #`, for example `nix develop .#node-env`. In a CI environment, though, it's usually better to run commands **_as if_** the shell environment were applied but without entering the environment (much like running `bash -c `). You can do that using the `--command` option. Here's an example: ```bash nix develop --command npm run build ``` If this command were run against a Nix shell environment with [npm](https://npmjs.org/) installed, the npm invocation would use the specific Nix-defined version instead of globally installed npm. This is the approach I use in my example project, as you'll see below. ## Using Nix inside your GitHub Actions pipeline So you've specified a Nix shell environment and maybe even used Nix to create some scripts that you intend to run both locally and in CI. Now you want to actually put that Nix logic to work in your pipelines. First, you need to install Nix. I've used two Actions for this, personally, and both work seamlessly: - [cachix/install-nix-action](https://github.com/marketplace/actions/install-nix) - [nixbuild/nix-quick-install-action](https://github.com/marketplace/actions/nix-quick-install) Here's an example pipeline that uses `cachix/install-nix-action` (using the `nixbuild` variant is effectively the same): ```yaml showLineNumbers steps: - uses: actions/checkout@v3 - name: Install Nix uses: cachix/install-nix-action@v17 with: # Mostly to avoid GitHub rate limiting extra_nix_config: | access-tokens = github.com=${{ secrets.GITHUB_TOKEN }} # Note: this would only work if Cargo is included in the Nix shell - name: Build release run: nix develop --command cargo build --release ``` For a more specific example, let's replace `creyD/prettier_action` with Nix logic. Let's say that we want to run `prettier --write **/*.{js,ts}` in our pipeline (to prettify all the JavaScript and TypeScript files in our repo). There are several ways to do this in a CI step. Let's start by using the `--command` flag: This is probably the fastest way forward. You know `prettier` is available in your environment so you run a command "sealed" inside your Nix shell environment. But there may be cases where you want to run scripts rather than raw commands. Here's an example of a script created with Nix using the [`writeScriptBin`][writeScriptBin] function: ```yaml showLineNumbers - name: Prettify JS and TS files run: nix develop --command prettier --write **/*.{js,ts} ``` To use that script in CI: ```nix showLineNumbers { prettify = pkgs.writeScriptBin "ci-prettify" '' prettier --write **/*.{js,ts} ''; } ``` Whether to use the `--command` flag or write Nix-based scripts is up to you. In my example project (discussed in the next section) I'll use raw commands instead of scripts for the sake of clarity. ## Example project To provide a more complete picture of the Nix-based approach I'm pushing for, I've created a project called [nix-github-actions](https://github.com/DeterminateSystems/nix-github-actions). It's a comically short [Rust](https://rust-lang.org/) "TODOs" web service and I kept it super basic because the song-and-dance _around_ the service is our focus here. Accompanying the code are several checks: - A [Rust formatting](https://github.com/DeterminateSystems/nix-github-actions/blob/main/rustfmt.toml) check - A check to make sure all the code in the repo conforms to the [EditorConfig](https://github.com/DeterminateSystems/nix-github-actions/blob/main/.editorconfig) - An [audit](https://github.com/DeterminateSystems/nix-github-actions/blob/main/deny.toml) of the Rust dependencies using [cargo-deny](https://github.com/EmbarkStudios/cargo-deny) - A general spell check using [codespell](https://github.com/codespell-project/codespell) If those checks pass, two things happen: - The Rust code is tested - The Rust code is built with the `--release` flag applied This is pretty standard fare for a Rust project on GitHub. But what makes this repo different is that there two identical GitHub Actions pipelines: - [`no-nix.yml`](https://github.com/DeterminateSystems/nix-github-actions/blob/main/.github/workflows/no-nix.yml) does things the way most repos do nowadays, using third-party Actions exclusively. To - [audit](https://github.com/DeterminateSystems/nix-github-actions/blob/main/.github/workflows/no-nix.yml#L36-L39) the Rust dependencies, for example, this pipeline uses [`EmbarkStudios/cargo-deny-action`](https://github.com/marketplace/actions/cargo-deny). - [`nix.yml`](https://github.com/DeterminateSystems/nix-github-actions/blob/main/.github/workflows/nix.yml) uses the Nix shell defined in flake.nix wherever possible. To give an example, instead of using a third-party Action like [`cargo-deny`](https://github.com/marketplace/actions/cargo-deny), this pipeline runs `nix develop --command cargo-deny check`. As I said above, one of the benefits of the Nix-based approach is that it's dramatically more straightforward to sync local dev environments with the CI environment because they _use the same environment_. If you'd like to run the CI checks from the repo on your machine (if you have Nix installed and flakes enabled): ```yaml showLineNumbers - name: Prettify JS and TS files run: nix develop --command ci-prettify ``` This [`ci-local`](https://github.com/DeterminateSystems/nix-github-actions/blob/main/flake.nix#L41-L61) script runs the entire CI suite, with the exception of installing Nix and [setting up a Rust cache](https://github.com/DeterminateSystems/nix-github-actions/blob/main/.github/workflows/nix.yml#L51-L60), and prints the result. With this Nix-built script available, you can avoid the vicious cycle you often confront with third-party Actions where you need to `git push` with empty or meaningless commit messages just to ensure that CI is working the way you expect. You can even set up scripts like this as [Git hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) if you like, although I haven't done that here. Some things to note from these contrasting pipelines: - Excluding the ubiquitous checkout Action, the "no Nix" pipeline uses five different third-party Actions while the Nix pipeline only uses two (one to [set up Nix](https://github.com/DeterminateSystems/nix-github-actions/blob/main/.github/workflows/nix.yml#L21-L25) and one to set up [caching](https://github.com/DeterminateSystems/nix-github-actions/blob/main/.github/workflows/nix.yml#L51-L60) for Rust). - The Nix-based pipeline runs faster in this repo. I won't make any bold generalizations about this, but when using Nix you do pay an up-front cost for building the shell environment, but that environment is then cached under `/nix/store` on the runner and running executables in that environment is quite "cheap" after the first run. By contrast, third-party Actions are indeed cached but they're cached as containers, which always brings overhead with it. ## Actions you should and shouldn't replace Before you go replacing _all_ of your Actions with Nix logic, I'd like to set forth some considerations. These types of Actions are good candidates for replacement: - `setup-*` actions like [`setup-java`](https://github.com/actions/setup-java) and [`setup-node`](https://www.notion.so/Streamlining-your-GitHub-Actions-dependencies-using-Nix-5dcf0dc66b884d7fa82efe9f696562cb). Nix shell environments make these totally superfluous because you can add whichever executables you want to your shell environment, including ones that aren't in [Nixpkgs](https://github.com/nixOS/nixpkgs) (if you create package definitions yourself). - Checkers, linters, and formatters. These pretty much _always_ fall firmly in the category of "wrapper around a single executable" and are thus trivially replaceable with Nix. Conversely, you're usually better off _not_ replacing any Actions that rely on lower-level APIs like the [Actions toolkit](https://github.com/actions/toolkit). Commonly used examples include [`cache`](https://github.com/actions/cache), [`upload-artifact`](https://github.com/actions/upload-artifact), and [`download-artifact`](https://github.com/actions/download-artifact). So if you need to interact with the Actions platform itself in a granular way, Nix-based CI logic is unlikely to be an improvement over the relevant third-party Actions. ## Implications I've focused on GitHub Actions in this post because it's widely used and because I don't always love its dependency system, but you can employ a similar Nix-based approach in just about any CI system. I strongly encourage you to at least explore the Nix approach for all of your CI use cases. After all, continuous integration is meant to automate your worries and troubles away. Every moment you spend fighting with CI is a moment that goes against the core purpose of the paradigm. With Nix you can reap all the benefits of CI while being unburdened of many common drawbacks. [writeScriptBin]: https://github.com/NixOS/nixpkgs/blob/f85482fdc8c128c5f2afff092fc3c522671f5648/pkgs/build-support/trivial-builders/default.nix --- **Make your QEMU 10 times faster with this one weird trick** Published: October 13, 2022 URL: https://determinate.systems/blog/qemu-fix [NixOS](https://nixos.org/) uses virtual machines based on [QEMU](https://www.qemu.org/) extensively for running its test suite. In order to avoid generating a disk image for every test, the test driver usually boots using a [Plan 9 File Protocol (9p)](http://man.cat-v.org/plan_9/5/intro) share (server implemented by QEMU) for the Nix store, which contains all the programs and config necessary for the test. I was working on a VM test that copied a fairly large amount of data (~278k files totaling ~5.3GiB) out of the 9p-mounted Nix store, and was surprised at how long copying this data took. On NVMe devices, I would expect this to take a matter of seconds or minutes, but the test actually ended up taking over 2 _hours_, most of which was spent copying files from 9p. Since this is untenably long for incremental work, I decided to dig into it a bit, and was able to reduce the duration of the test to only 7 minutes. In this post, I'll describe the whole journey. ## Profiling QEMU As a preface: I don't have much experience in debugging performance issues! Most of what I used was novel to me. The first thing I wanted to do was find out where a lot of time was being spent. My guess was that it was in QEMU and not in the guest, though this guess being correct was a matter of pure luck. [This stack overflow question](https://stackoverflow.com/questions/16999681/how-to-do-profiling-with-qemu) described a problem mostly but not entirely unlike mine. This led me down the wonky path of trying to use the [poor man's profiler](http://poormansprofiler.org/), a little hack composed of gdb, some shell, and awk. ## Stories of surprises and failure: poor man's profiler I immediately ran into a minor roadblock with this approach. gdb said: ``` warning: Target and debugger are in different PID namespaces; thread lists and other data are likely unreliable. Connect to gdbserver inside the container. ``` Nix uses Linux namespaces to provide builds with some isolation from the system running the build in order to reduce the effects that the environment of a particular machine can have on the result of the build ("purity"). This includes PID namespaces, which prevent processes within the namespace from touching any processes outside the namespace. gdb was unhappy with being in a different PID namespace from the process it was targeting! I first attempted to get my gdb inside the sandbox using nsenter. The first surprise I encountered here was that entering a PID namespace does _not_ cause utilities from procps, such as ps, pgrep and top, to report only on processes inside the new namespace: ```bash [root@oak:~]# pgrep -a qemu 1678991 /nix/store/6shk4z9ip57p6vffm5n9imnkwiks9fsa-qemu-host-cpu-only-for-vm-tests-7.0.0/bin/qemu-kvm [...] [root@oak:~]# nsenter --target=1678991 --pid 🗣 This spawned a new shell within the build's PID namespace [root@oak:~]# ps -f 1 UID PID PPID C STIME TTY STAT TIME CMD root 1 0 0 Sep02 ? Ss 1:24 /run/current-system/systemd/lib/systemd/systemd ``` What!? That's not the PID1 I'm expecting! And I certainly haven't been running this build since the 2nd of September. I'll omit the details of the hour of frustration that ensued, but it turns out that the `/proc` which `ps` and friends were reading from was still that of the root PID namespace—even though they were no longer running in it! This might cause some funny unexpected behaviour when using tools like `pkill`… But that's a problem for another day. With my newfound knowledge, I was able to work around this issue by also creating a new mount namespace and mounting the desired `proc` filesystem on top of the `/proc` we inherited from the root namespace. ```bash [root@oak:~]# nsenter -t 1684606 -p -- unshare -m 🗣 Now inside the build's PID namespace (through nsenter) and a new mount namespace (created by unshare) [root@oak:~]# ps -f 1 UID PID PPID C STIME TTY STAT TIME CMD root 1 0 0 Sep02 ? Ss 1:24 /run/current-system/systemd/lib/systemd/systemd [root@oak:~]# mount -t proc proc /proc [root@oak:~]# ps -f 1 UID PID PPID C STIME TTY STAT TIME CMD nixbld1 1 0 0 12:27 ? Ss 0:00 bash -e /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh [root@oak:~]# ps -ef UID PID PPID C STIME TTY TIME CMD nixbld1 1 0 0 12:27 ? 00:00:00 bash -e /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh nixbld1 6 1 0 12:27 ? 00:00:00 /nix/store/pn7863n7s2p66b0gazcylm6cccdwpzaf-python3-3.9.13/bin/python3.9 /nix/store/kdi82vgfixayxaql77j3nj7 nixbld1 7 6 99 12:27 ? 00:04:00 /nix/store/6shk4z9ip57p6vffm5n9imnkwiks9fsa-qemu-host-cpu-only-for-vm-tests-7.0.0/bin/qemu-kvm -cpu max -na root 46 0 0 12:29 pts/5 00:00:00 -bash root 79 46 0 12:30 pts/5 00:00:00 ps -ef 🗣 That's better! [root@oak:~]# pid=7 ./pprof 1500 __futex_abstimed_wait_common,__new_sem_wait_slow64.constprop.1,qemu_sem_timedwait,worker_thread,qemu_thread_start,start_thread,clone3 743 __lll_lock_wait,pthread_mutex_lock@@GLIBC_2.2.5,qemu_mutex_lock_impl,qemu_mutex_lock_iothread_impl,flatview_read_continue,flatview_read,address_space_rw,kvm_cpu_exec,kvm_vcpu_thread_fn,qemu_thread_start,start_thread,clone3 100 syscall,qemu_event_wait,call_rcu_thread,qemu_thread_start,start_thread,clone3 53 ioctl,kvm_vcpu_ioctl,kvm_cpu_exec,kvm_vcpu_thread_fn,qemu_thread_start,start_thread,clone3 45 get_fid,v9fs_read,coroutine_trampoline,__correctly_grouped_prefixwc,?? 15 get_fid,v9fs_walk,coroutine_trampoline,__correctly_grouped_prefixwc,?? 11 alloc_fid,v9fs_walk,coroutine_trampoline,__correctly_grouped_prefixwc,?? 8 get_fid,v9fs_getattr,coroutine_trampoline,__correctly_grouped_prefixwc,?? 5 clunk_fid,v9fs_xattrwalk,coroutine_trampoline,__correctly_grouped_prefixwc,?? 5 alloc_fid,v9fs_xattrwalk,coroutine_trampoline,__correctly_grouped_prefixwc,?? 4 __lll_lock_wait,pthread_mutex_lock@@GLIBC_2.2.5,qemu_mutex_lock_impl,qemu_mutex_lock_iothread_impl,flatview_write_continue,flatview_write,address_space_rw,kvm_cpu_exec,kvm_vcpu_thread_fn,qemu_thread_start,start_thread,clone3 3 get_fid,v9fs_xattrwalk,coroutine_trampoline,__correctly_grouped_prefixwc,?? 3 get_fid,v9fs_open,coroutine_trampoline,__correctly_grouped_prefixwc,?? 2 clunk_fid,v9fs_clunk,coroutine_trampoline,__correctly_grouped_prefixwc,?? 1 get_fid,v9fs_readlink,coroutine_trampoline,__correctly_grouped_prefixwc,?? 1 get_fid,v9fs_readdir,coroutine_trampoline,__correctly_grouped_prefixwc,?? 1 address_space_translate_internal,flatview_do_translate.isra,address_space_map,virtqueue_map_desc,virtqueue_split_pop,virtio_blk_handle_vq,virtio_queue_notify_vq.part,aio_dispatch_handler,aio_dispatch,aio_ctx_dispatch,g_main_context_dispatch,main_loop_wait,qemu_main_loop,main 1 ``` 1. Locking resources, at 1500 and 743 thread samples. Since the top result was on worker threads, I guessed that wasn't interesting and they were just waiting for work to become available. 2. Waiting for things to happen inside the VM, at 100 and 53 thread samples. That didn't seem like a relevant issue though—I'd expect a VM monitor to be waiting for its guest to need something most of the time. The rest of the numbers were small enough that I (erroneously!) considered them uninteresting too. At this point I became frustrated enough with this branch of my investigation that I gave up on it. I moved on to [quickstack](https://github.com/yoshinorim/quickstack/), [packaged it](https://github.com/NixOS/nixpkgs/pull/190335), and observed that it couldn't get me any information on threads, before going all the way back to the Stack Overflow question to follow [the other link](https://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html) provided in the answer. ## Flame graphs with perf This was what really got me somewhere. After recording performance data using `perf record -F max -a -g -- sleep 20` while the build was running, I was able to generate a flame graph which made the source of the performance problems quite apparent. This command: ```bash perf script | stackcollapse-perf.pl | # taken from the flamegraph page linked above grep ^qemu | # We're only interested in the qemu process awk '/unknown/ { gsub("(\\[unknown];){1,}", "[unknown...];", $0) } { print }' | # Make the graph a lot less tall by collapsing multiple consecutive unknown stack frames together flamegraph.pl > flamegraph.svg # and generate a flamegraph ``` produced this nifty interactive SVG graph: {/* import QemuFlamegraphSvg from "~/assets/qemu-flamegraph.svg"; */} The prevalence of "fid"-related functions is quite apparent in this graph. So I jumped into the 9p docs and QEMU source code to find out that fids are numbers which refer to open files in a 9p connection, similar to file descriptors in the POSIX file API—so there is one fid for every file that the guest has open. Let's look at the previous implementation of `get_fid`, which QEMU's 9p server uses in implementations of `stat` (getting file metadata), `open` (opening a file), and `read` (reading data from an open file), amongst others: ```c showLineNumbers static V9fsFidState *coroutine_fn get_fid(V9fsPDU *pdu, int32_t fid) { int err; V9fsFidState *f; V9fsState *s = pdu->s; // Blog note: I've omitted some parts that are irrelevant to performance here. QSIMPLEQ_FOREACH(f, &s->fid_list, next) { if (f->fid == fid) { return f; } } return NULL; } ``` QEMU iterates over `fid_list` to find the desired `fid`'s data. Finding an entry in a list by iterating over has a time complexity of `O(n)` where `n` is the size of the list—in this case, the list of _all the files the guest has open_—so this is expensive! Moreover, some inspection of QSIMPLEQ (`QEMU simple queue`) reveals that it's implemented as a linked list, a data structure which tends to exhibit poor cache locality. Since my test copies many files from the 9p filesystem, as well as being booted from it, this lookup happens quite often: - One `stat` for getting a file's metadata (permissions in particular) - One `open` to get a handle on a file - One `read` for small files, or many for larger files, to get the contents of the file That makes at least 3 operations which perform the lookup, for each of the 278000 files, which bring the inefficient lookup into a hot code path. _This_ was the reason for the slowness. ## Fixing it What we really want is a structure where we can look up entries by `fid` more cheaply. We can't just use an array-based vector, which would consistently give us `O(1)` lookup, because `fid`s are chosen by the client: we can't rely on every newly allocated `fid` just being the smallest unoccupied one, and need to support arbitrary 32-bit integers. I opted for a hash table, as conveniently [implemented by glib](https://docs.gtk.org/glib/struct.HashTable.html), which QEMU already depends on. That provides us with `O(1)` best-case complexity, while keeping the worst case at `O(n)`. The exact real-world performance characteristics are significantly more complex than with the linked list, and there may be marginally more suitable data structures (or hash table implementations) out there, but we're looking for a big win and not micro-optimisation here. Rewriting the relevant code was surprisingly uneventful, to the point that once I had it compiling, it just worked (an experience I'm not sure I've ever had before with C!). The results were spectacular: my previously >2h test finished in 7 minutes. It also reduces the build time of NixOS's ZFS AWS image from 19 minutes to 1. It was pretty clear to me that this needed to go upstream. ## Contributing the fix QEMU uses an [email-based workflow](https://www.qemu.org/docs/master/devel/submitting-a-patch.html), where patches are sent as emails to a mailing list which maintainers are subscribed to. I hadn't done this before, and it went somewhat chaotically, as you can see by looking at the threads ([v1](https://lists.nongnu.org/archive/html/qemu-devel/2022-09/msg00472.html) [v3](https://lists.nongnu.org/archive/html/qemu-devel/2022-09/msg01266.html) [v1a](https://lists.nongnu.org/archive/html/qemu-devel/2022-09/msg04051.html) [v5](https://lists.nongnu.org/archive/html/qemu-devel/2022-09/msg04575.html) [v6](https://lists.nongnu.org/archive/html/qemu-devel/2022-10/msg00370.html)) in the list archives. There's a lot of space for mistakes in email-based patch submission, and I made several: - Forgetting to reply-all when answering review comments, so that the reply is visible to everyone interested and not just the reviewer - Missing the version tag on a resubmission (in my case, this was because of [misleading docs](https://lists.nongnu.org/archive/html/qemu-devel/2022-09/msg01795.html)) - Sending a resubmission as a reply to the previous thread (that was just me failing to read the docs) - Losing a bunch of patch-documenting work while using git-publish because `git send-email` failed (nixpkgs does not enable Git's email functionality in the default package, and git-publish unconditionally deletes the working files once the send command has run). But the reviewers, especially Christian Schoenebeck (thanks!) were helpful and patient, and in the end [the patch made it](https://lists.nongnu.org/archive/html/qemu-devel/2022-10/msg00909.html) (though an apparently unrelated bug was discovered) and will soon be finding its way to a QEMU version near you! If you can't wait and you use Nix, you can pull the patch in with this overlay: ```nix showLineNumbers final: prev: { qemu = prev.qemu.overrideAttrs (o: { patches = o.patches ++ [ (prev.fetchpatch { name = "qemu-9p-performance-fix.patch"; url = "https://gitlab.com/lheckemann/qemu/-/commit/ca8c4a95a320dba9854c3fd4159ff4f52613311f.patch"; sha256 = "sha256-9jYpGmD28yJGZU4zlae9BL4uU3iukWdPWpSkgHHvOxI="; }) ]; }); } ``` ## Outcomes While the core of what I changed wasn't hugely complex, this adventure had many collateral (positive!) outcomes for me: - I wrote Nix packages for [quickstack](https://github.com/NixOS/nixpkgs/pull/190335) (including an upstream PR that makes the build more generic) and [git-publish](https://github.com/NixOS/nixpkgs/pull/190309). - I used `perf` to work on performance issues for the first time, learning how useful it is and basics of how to use it. - I learnt that different methods of profiling can yield wildly different results—the results from perf painted a different picture from the poor man's profiler; I don't understand why yet, but I'll spend some more time learning about them soon. - I submitted my first patch to QEMU, getting to know the code a little. - I submitted my second patch to QEMU (trying to improve the patch submission docs). - I learnt how to submit patches by email, and which mistakes to avoid. - I learnt about the 9p protocol. - My tests now run much faster! - And I got the warm fuzzy feeling of having made a valuable contribution. My takeaway from it? Digging into a frustrating problem—even if I'm totally unfamiliar with the code involved and the technology I'm using—can be hugely rewarding! Not only have I made my own work on the tests a lot more pleasant by reducing turnaround time, but this will go out to _all_ QEMU users, and significantly reduce the load generated by installer tests on NixOS's build farm. This is what I love about working on open source software: if something's wrong, I have the means to go in there and fix it, and can pass the benefits on to everyone else who uses it. It doesn't always go as well as this, but this kind of experience can make a great many less successful adventures worth it. _Also posted on [Linus's own blog](https://linus.schreibt.jetzt/posts/qemu-9p-performance.html)_ --- **Using Nix to run software with no installation steps** Published: October 7, 2022 URL: https://determinate.systems/blog/nix-run While [Nix](https://nixos.org/) is most widely known for its core features, like fully reproducible [package builds](https://nix.dev/manual/nix/stable/package-management) and hermetic [development environments](https://nixos.org/guides/declarative-and-reproducible-developer-environments.html), it has numerous features that are less fundamental to its value proposition but nonetheless _extremely_ useful. Today, I'd like to bring one such Nix feature into focus: Nix's ability to run executables with zero setup and nothing more than a URL in hand. Here's an example: ```bash nix run github:DeterminateSystems/riff ``` If you have Nix installed and [flakes enabled](https://nixos.wiki/wiki/Flakes), this one command enables you to run [Riff](https://riff.sh/) in one go, without needing to run `nix profile install` or `apt-get` or `brew install` or anything else. There's really nothing quite like it in the software world, and I'm excited to let you in on this little secret. ## The core mechanism: Nix flakes The `nix run` feature only works with Nix [flakes](https://nixos.wiki/wiki/Flakes), that is, with any Nix project with a [properly structured](https://nixos.wiki/wiki/Flakes#Flake_schema) `flake.nix` file in the root.. There are two Nix flake [outputs](https://nixos.wiki/wiki/Flakes#Output_schema) that you can directly `nix run`: packages and apps. ### Nix packages While flake package outputs are typically used to build final artifacts, such as `nix build .#my-app`, you can also `nix run` those outputs if \*\*the built derivation has a `bin` directory with an executable in it. If so, Nix tries these executable names in order: - The [`meta.mainProgram`](https://nixos.org/manual/nixpkgs/stable/#var-meta-mainProgram) attribute of the derivation - The [pname](https://nixos.org/manual/nixpkgs/stable/#sec-package-naming) attribute of the derivation - The [name](https://nixos.org/manual/nixpkgs/stable/#sec-package-naming) attribute of the derivation If none of those are successful, then you can't `nix run` the package output. The command `nix run github:DeterminateSystems/riff` works, for example, because the default package output has [pname](https://github.com/DeterminateSystems/riff/blob/main/flake.nix#L97) set to `riff`. ### Nix apps Nix apps are executable programs that you can specify using an attribute set with two fields: ```nix showLineNumbers { apps.default = { type = "app"; program = "${myPkg}/bin/my-pkg"; # Assuming there's a local derivation called myPkg }; } ``` If you add the above to your flake outputs, then you can run the app using `nix run` in the current directory or `nix run ` if `flake.nix` is somewhere else. Because this is the default app, you don't need to specify an app name like `nix run .#my-app`. To specify multiple apps in a flake: ```nix showLineNumbers { apps = { my-linter = { type = "app"; program = "${myLinter}/bin/my-linter"; }; my-checker = { type = "app"; program = "${myChecker}/bin/my-checker"; }; }; } ``` To run these apps: ```bash nix run #my-linter nix run #my-checker # If the flake were in the current directory nix run .#my-linter nix run .#my-checker # If the flake were in a GitHub repo nix run github:my-org/my-repo#my-linter nix run github:my-org/my-repo#my-checker ``` Using Nix apps for `nix run` instead of packages is especially beneficial when a single derivation builds multiple executables. Here's an example: ```nix showLineNumbers { apps = { # nix run #server server = { type = "app"; program = "${myPkg}/bin/server"; }; # nix run #client client = { type = "app"; program = "${myPkg}/bin/client"; }; }; } ``` ## Full example Let's dig in a bit more concretely. Let's say that you're working on a tool called runme in a public Git repo at https://github.com/AmbiguousTechnologies/runme. Your `flake.nix` looks like this: ```nix showLineNumbers { description = "runme application"; inputs = { nixpkgs.url = "nixpkgs"; # Resolves to github:NixOS/nixpkgs # Helpers for system-specific outputs flake-utils.url = "github:numtide/flake-utils"; }; outputs = { self, nixpkgs, flake-utils }: # Create system-specific outputs for the standard Nix systems # https://github.com/numtide/flake-utils/blob/master/default.nix#L3-L9 flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { # A simple executable package packages.default = pkgs.writeScriptBin "runme" '' echo "I am currently being run!" ''; # An app that uses the `runme` package apps.default = { type = "app"; program = "${self.packages.${system}.runme}/bin/runme"; }; }); } ``` With that configuration in place and pushed to the repo, you could run the runme script with just one command: ```bash nix run github:AmbiguousTechnologies/runme ``` In this case, runme is the default app so nothing needs to come after the flake URL, but let's say that you add a second app, called `runme-lint`: ```nix showLineNumbers { apps = rec { default = runme; runme = { type = "app"; program = "${self.packages.${system}.runme}/bin/runme"; }; lint = { type = "app"; program = "${self.packages.${system}.runme-lint}/bin/runme-lint"; }; }; } ``` You can now run both of those apps: ```bash nix run github:AmbiguousTechnologies/runme nix run github:AmbiguousTechnologies/runme#runme # Equivalent to the above nix run github:AmbiguousTechnologies/runme#runme-lint ``` An important thing to note here is that these `nix run` invocations require _zero_ installation or setup. Here's what happens instead: - Nix inspects the `program` string in each app and determines that it needs to build a new package. - Nix builds the package and all of its dependencies and stores the results in the local Nix store (by default under `/nix/store`). - Once the package has been built, Nix can run it directly from the local store. In this case, the `runme` package may end up in a location like `nix/store/khid71caxbq4g8dhfln8rihyclmrz7c3-runme`. But the installation process happens seamlessly and without any user input. This multi-step process accounts for why the first `nix run` invocation usually takes a while but is near-instantaneous on future runs. If you opt to use `nix run` for running packages—instead of installing them using `nix profile install` or [Home Manager](https://github.com/nix-community/home-manager) or some other means—you should take this potential time lag into account. ## Using Git revisions as a versioning mechanism For Nix flakes that are Git repositories, you can point `nix run` at specific Git revisions to run different versions of a package or app. Here's the basic syntax for specific revisions on GitHub, along with some examples: ```bash # Syntax nix run github://# # Examples ## Specific commit ID nix run github:DeterminateSystems/riff/a71a8b5ddf680df5db8cc17fa7fddd393ee39ffe ## Tag nix run github:DeterminateSystems/riff/v1.0.0 ## Latest commit in a branch nix run github:DeterminateSystems/riff/secret-branch-for-nix-run ## Target a flake in a subdirectory nix run "github:hard-to-find/cool-app?dir=nested#specific-app" ``` Let's say that you've released a great new tool called quicknav at version 0.1.0. You've worked out the kinks and it's stable enough to offer to the world. But late at night you implement a wacky idea for the UI that's definitely _not_ yet ready for a stable release, but you think your friends would be entertained by it. You want them to try it out. What to do? With Nix, one possibility is to push those changes to separate branch, let's say `experimental-ui`, and let your friends run that: ```bash nix run github:quicknav/quicknav/experimental-ui ``` In this case, `github:quicknav/quicknav` and `github:quicknav/quicknav/experimental-ui` aren't "versions" of one flake—they're really just _different_ flakes. In other words, Nix understands both of those addresses as different "universes" of Nix code with no necessary connection to each other. The same applies to _all_ Git refs. ## Security warning It bears noting here that `nix run` has a pretty loose trust model. Running `nix run github:DeterminateSystems/riff` today and then again tomorrow, for example, could execute entirely different executables because the flake address (github:DeterminateSystems/riff) could resolve to different Git commits at different times. If you're concerned about this problem—and you should be!—the best course of action is to always tie your `nix run` invocations to specific commits or to Git refs that resolve to a known commit. Below you'll find a bad `nix run` and a good `nix run`: ```bash # 🚫 BAD nix run github:kinda-sketch/could-be-a-bitcoin-miner # ✅ BETTER (if you've already vetted the code for this commit) nix run github:kinda-sketch/could-be-a-bitcoin-miner/0a8e7a241db7938b10062218309ba30f6d9f0e2d ``` Granted, worrying about Git refs negates much of the "a-ha!" effect of `nix run`, but if you plan on using this feature with less-than-100%-trusted remote flakes, it's nonetheless advised. ## How to find out which `nix run` targets are available With the Nix CLI, you can run `nix flake show ` to see what a flake outputs. Let's see which packages are output by the flake in the [Riff](https://riff.sh/) repo: ```bash nix flake show github:DeterminateSystems/riff ``` The output: ``` github:DeterminateSystems/riff/a54624ac12aa2c125d8891d124519fba17aa2016 │├───defaultPackage │ ││ ├───aarch64-darwin: package 'riff-1.0.2' │ ││ ├───aarch64-linux: package 'riff-1.0.2' │ ││ ├───x86_64-darwin: package 'riff-1.0.2' │ ││ └───x86_64-linux: package 'riff-1.0.2' # Other output ``` From this we can see that Riff's flake outputs a default package on four different platforms. When you run `nix run`, Nix infers the current host platform (in my case `aarch64-darwin`) and runs the system-specific package if it's available or errors out if it isn't available. ## Implications To my knowledge, no software ecosystem outside of Nix provides a comparably baked-in mechanism for running executables. Containerization tools like [Docker](https://docker.com/) and [Podman](https://podman.io/) get you somewhere in the ballpark by enabling you to use `docker run` and `podman run` to target OCI-compliant images in container registries, but both of those tools require that the image be already built and pushed, whereas Nix needs nothing more than a properly formed Nix expression exposed through a known flake URL. While I wouldn't necessarily call this a primary, bread-and-butter use case for Nix, I do think that it provides a vivid illustration not just of what Nix can do but also of the power of the Nix flake model for distributing expressions. With Nix, you can often get things done without needing to distribute build artifacts. All you need is valid Nix code and the Nix CLI can build—and run—arbitrarily complex artifacts on the current host. And with flakes providing convenient "addresses" for Nix code, you end up with a robust mechanism for distributing executables. --- **An invitation to Rust maintainers** Published: October 5, 2022 URL: https://determinate.systems/blog/riff-rust-maintainers A few weeks ago, we at [Determinate Systems](https://determinate.systems) released [Riff](https://riff.sh), a tool that uses Nix to automatically provide **external dependencies** for Rust projects. By that we mean dependencies written in other languages—like [OpenSSL](https://www.openssl.org/) or the [Protobuf](https://developers.google.com/protocol-buffers) compiler—that by definition can't be installed by Cargo. The developer experience of [Rust](https://rust-lang.org) is widely regarded as one of the best in the business—and we concur!—but external dependencies have been a thorn in the side of the Rust community from the get-go. And so we're excited to provide a full-fledged solution to this problem in the form of [Riff](https://riff.sh). The response thus far has been quite enthusiastic, with users on [Twitter](https://twitter.com/DeterminateSys/status/1567196954093883392), [Reddit](https://www.reddit.com/r/rust/comments/x7gd6v/introducing_riff_a_nixbased_tool_for/), [Hacker News](https://news.ycombinator.com/item?id=32739954), and other forums expressing variations of "magic!" and "finally!" (mixed with some quite reasonable critique, of course). We'd like to follow up on the initial launch by **calling upon Rust project maintainers to support Riff in their projects**. As you'll see in this post, supporting Riff is non intrusive, and of benefit not just to devs that directly depend on your project but rather to the whole Rust ecosystem. ## How to support Riff in your Rust project To give an example, let's say that we're creating a game engine in Rust called **gam3r**. Like many game engines, gam3r has some external—that is, non-Rust—dependencies that need to be installed in the current environment. More specifically: - On all systems, `protoc`, the [Protocol Buffers](https://developers.google.com/protocol-buffers) compiler, needs to be installed at build time. - On macOS systems, both Intel (`x86_64`) and Apple Silicon (`aarch64`), Apple's [`CoreServices`](https://developer.apple.com/documentation/coreservices) framework needs to be installed. Adding this to your `Cargo.toml` config would notify Riff to install these dependencies: ```toml showLineNumbers [package.metadata.riff] build-inputs = ["protobuf"] [package.metadata.riff.targets.aarch64-apple-darwin] build-inputs = ["darwin.apple_sdk.frameworks.CoreServices"] [package.metadata.riff.targets.x86_64-apple-darwin] build-inputs = ["darwin.apple_sdk.frameworks.CoreServices"] ``` With this metadata in place, Riff knows that it needs to install these tools not just in gam3r but also in any project that depend on gam3r, no matter how far down the dependency tree gam3r is. The implication is that the more depended upon gam3r becomes, the more beneficial our Riff metadata becomes. For more in-depth instructions on adding Riff metadata, see our [ official docs ](https://www.riff.sh/docs/rust/#how-to-declare-external-dependencies). If we neglect to provide that metadata, commands like `riff run cargo build` are far more likely to fail on downstream projects, shifting the burden to maintainers of those projects to provide Riff metadata. But adding this small handful of lines to our `Cargo.toml` improves the chances that `riff run` and `riff shell` on downstream projects will Just Work™️, thereby eliminating those small bits of misdirected time and effort that really add up on an ecosystem level. ## Our goal Our end goal is to enable Riff to build and/or run **100% of Rust projects**. Is that truly achievable? This will depend on you all as Rust maintainers, of course, but we're confident that Riff, with proper buy-in, has the potential to positively obliterate the external dependency problem in Rust (and in other languages later). We believe that every time a Rust project with external dependencies—and that's the lion's share of heavily depended-upon Rust projects—is made "Riffable," the already-thriving Rust ecosystem takes a small step forward and asymptotically approaches a state where `riff shell` and `riff run` become second nature. We cordially invite all Rust maintainers to help us get there. --- **Effortless dev environments with Nix and direnv** Published: September 23, 2022 URL: https://determinate.systems/blog/nix-direnv Like many of you, I work on a lot of different projects. Even when a project is less serious—hey, I should check out this new JS framework!—I strive to reduce the friction involved with setting up the project's dev environment to the absolute bare minimum possible. In this post, I'd like to tell you how I do that using two tools that have become indispensable to my daily workflows: [Nix](https://nixos.org/) and [direnv](https://direnv.net/). ## Prerequisites If you'd like to follow along with this tutorial: - [Enable flakes on NixOS](https://nixos.wiki/wiki/Flakes#Enable_flakes) or install [Nix with Flakes enabled on any other Linux and macOS](/blog/determinate-nix-installer). - Install [direnv](https://direnv.net). I personally use [Home Manager](https://github.com/the-nix-way/nome/blob/e60270c03aaa167e8d0b761632a39f640dfec05c/home/programs.nix#L16-L21) to install this but of course feel free to use whichever method works for you. Optionally, you can also install [nix-direnv](https://github.com/nix-community/nix-direnv), which provides a faster implementation of the Nix-related functionality in direnv. ## Nix shell environments Before I go deeper into the fearsome combo of Nix and direnv, I'd like to say a bit about the [Nix shell](https://nix.dev/tutorials/declarative-and-reproducible-developer-environments) for those who aren't yet familiar. [Nix](https://nixos.org) enables you to create fully declarative and reproducible development environments that are defined by: - Which Nix packages you want installed in the environment - Which shell commands you want to run whenever the environment is initialized The standard function for creating shell environments is [`mkShell`](https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-mkShell) in Nixpkgs, but other functions are available in the ecosystem. Here's an example: ```nix showLineNumbers { devShells.default = pkgs.mkShell { buildInputs = with pkgs; [go_1_19 terraform]; }; } ``` When you enter this shell by running `nix develop`, Go version 1.19 and Terraform will both be available in the current directory (but not globally). With definitions like this, you can include every package you may need in the environment, which includes executables (like Go and Terraform) as well as things like language servers and editor configurations. ## direnv and Nix environments [direnv](https://direnv.net/) is a tool that, whenever you navigate to a directory with a `.envrc` file in it, enacts the directives in that file to produce a directory-specific environment _every time_ you navigate there. direnv has a lot of great features worth checking out, such as [loading `.env` files](https://direnv.net/man/direnv-stdlib.1.html#codedotenv-ltdotenvpathgtcode) and [managing local Ruby versions](https://direnv.net/man/direnv-stdlib.1.html#codeuse-rbenvcode), but today I want to focus on just one: the [`use flake`](https://github.com/direnv/direnv/blob/master/stdlib.sh#L1256-L1271) directive. If you add this to your `.envrc` file, direnv activates the [Nix shell](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-develop.html#flake-output-attributes) defined in the local `flake.nix`: ```bash use flake ``` Practically, this means that all you need to do to enter your Nix-defined development environment is to `cd` into the directory (unless you run `direnv revoke`, which causes direnv to ignore the `.envrc` until it's reactivated using `direnv allow`). This is an elegant solution to a problem that we've all encountered where we navigate to a directory, run a setup command, and end up confused because the desired executable is missing or some other aspect of the environment isn't what we want. ## direnv with remote Nix environments The `use flake` directive works great when the `.` points to a local flake defined by a `flake.nix` file, automatically loading the dev shell defined under the `devShells.default` output. But there's actually an even faster way to use direnv for dev environments because `use flake` can point to _any_ dev shell defined in any flake. Take this [Node.js dev environment](https://github.com/the-nix-way/dev-templates/blob/main/node/flake.nix#L27-L33) as an example: ```nix showLineNumbers { # Inputs omitted for brevity outputs = { ... }: { devShells.default = pkgs.mkShell { packages = with pkgs; [node2nix nodejs pnpm yarn]; shellHook = '' echo "node `${pkgs.nodejs}/bin/node --version`" ''; }; }; } ``` The URL for this flake is `github:the-nix-way/dev-templates?dir=node`, so if we want to use this environment, we can add this to the `.envrc` file instead: ```bash use flake "github:the-nix-way/dev-templates?dir=node" ``` If you run `direnv allow` in the directory, direnv gets to work using Nix to install the `packages` defined in this remote flake (Node.js, npm, etc.). That can take a while time the first time you enter the directory because Nix needs to install the environment's dependency tree into the Nix store, but when you `cd` in the future it should be quite speedy. But the potential fun doesn't stop there. I've created, for example, my own little helper script for creating `.envrc` files that I call `dvd`: ```bash echo "use flake \"github:the-nix-way/dev-templates?dir=$1\"" >> .envrc direnv allow ``` So if I run .dvd go., for example, my shell immediately starts loading [this environment](https://github.com/the-nix-way/dev-templates/blob/main/go/flake.nix#L22-L37), which includes Go version 1.19 and some commonly used Go tools (like [goimports](https://pkg.go.dev/golang.org/x/tools/cmd/goimports)). No Dockerfile, no Makefile, no local Nix logic, just a single line in a `.envrc` file can get you a specific, arbitrarily complex dev environment. ## Layering environments Because `.envrc` files are scripts, you can add as many `use flake` statements as you want, which enables you to _layer_ Nix environments. Here's an example `.envrc`: ```bash use flake "github:the-nix-way/dev-templates?dir=elixir" use flake "github:the-nix-way/dev-templates?dir=gleam" ``` In this case, both an [Elixir](https://github.com/the-nix-way/dev-templates/blob/main/elixir/flake.nix) and a [Gleam](https://github.com/the-nix-way/dev-templates/blob/main/gleam/flake.nix) environment would be applied to the current directory. This may not _always_ be the best idea, as Nix honors the flake listed last in case of a clash, for example in the case of two packages named `cargo` that refer to different versions of [Cargo](https://doc.rust-lang.org/cargo/). But if you apply due care, you can reap the benefits of layered environments without being stung by ambiguity. ## Drawback The major pro of using remote flakes for dev environments is that it enables you to initialize an isolated, pure dev environment about as quickly as you can hope for in the software world. For weekend projects and experiments it's perfect. The downside is that relying on a dev environment defined in a remote flake doesn't provide exactly what you need. Layering environments can add some specificity but for projects that require a fully custom environment, the approach in this post breaks down pretty quickly. But never fear! There's another great Nix feature that can help here: [Nix flake templates](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-init.html). With templates, you can use the `nix flake init` command to initialize a Nix-defined template in the current directory. Personally, I use a script called `dvt` to initialize specific templates from my [dev-templates] project: ```bash nix flake init -t "github:the-nix-way/dev-templates#$1" direnv allow ``` So if I run dvt I get a specific `.envrc`, `flake.nix`, and `flake.lock`. These files provide a great basis for further customization. Let's say that I need to start up a Python project but I also want to install [watchexec]. For that I can run `dvt python`, wait for direnv to load the environment, and then add `watchexec` to the `packages` in `flake.nix`. ## Security warning One thing that you should always bear in mind when using remote flakes for dev environments is that you can add _anything_ to `packages` and that `shellHook` allows for arbitrary code execution. If you add `use flake "github:bitcoin-scam/gonna-mine-btc"` to your `.envrc` you are going to be in for a rough time (and you could still be in for a rough time if you use a more innocuous-seeming flake). So there are some precautions you should always take: 1. Examine the actual contents of the flake to make sure nothing funny is going on. 2. Less obviously, if you're using a version-controlled remote flake, make sure to "pin" to a specific reference. So instead of `use flake "github:the-nix-way/dev-templates?dir=rust"` you can use `use flake "github:the-nix-way/dev-templates/b2377c684479f6bf7d222a858e1eafc8a2ca3499?dir=rust"` to make the env truly declarative and overcome the ambiguity of the first `use flake` directive. ## Broader implications Using techniques like this has helped me to not just get up and going in dev projects more quickly than ever before—seriously, it almost feels like cheating—but also to declutter my home environment. I talked about this a little bit in a [previous post](/blog/nix-home-env), where I laid out an imperative that I've been following recently where I keep my laptop's global environment as minimal as possible—just Git, jq, wget, and a few other executables that I truly need available everywhere on my system—while making everything else project specific. As devs, we should spoil ourselves more. We should get used to dropping into projects willy-nilly and having them Just Work, and the processes that make them Just Work should fade into the background. The approach I laid out here charts what I take to be a reasonable course between convention and configuration. I certainly don't expect you to make the same choices, but I do hope that the formidable combination of Nix flakes and direnv becomes an extra-sharp arrow in your daily dev quiver. [dev-templates]: https://github.com/the-nix-way/dev-templates [watchexec]: https://github.com/watchexec/watchexec --- **Building a highly optimized home environment with Nix** Published: September 15, 2022 URL: https://determinate.systems/blog/nix-home-env A few weeks ago, fresh off of a minor victory in my local dev setup, I [tweeted](https://twitter.com/lucperkins/status/1555138195507105794) this on a whim: The reaction—120+ likes and 20+ retweets—was modest by broader Twitter standards but pretty strong for a rather vague tweet about [Nix](https://nixos.org). Today, I'd like to follow up on this tweet and show how I've used Nix to streamline my laptop environment in ways that have saved me time and made me substantially more productive across programming languages and platforms—and even jobs. ## How I used to do things Once upon a time, I managed my home environment, across several laptops and one desktop, the way that many devs do nowadays: I maintained a series dotfiles (for Vim, tmux, and others) that I updated now and then as needs changed. For executables, I used `brew install` whenever the need arose (I'm a macOS user almost exclusively). Whenever I needed to change machines—new work laptop, new present to myself—I eagerly looked forward to building an environment I liked from scratch and in an ad-hoc way, usually with the help of my `dotfiles` Git repo and some kludgy shell scripts. This approach always more or less "worked" but it was always heavy on time and cognitive effort. The words "reproducible" and "declarative" weren't yet on my radar, at least in this domain, but my longing for them certainly was. And then a piercing ray of light came into my life. ## Enter Home Manager When I first came upon Nix eight years ago, I was intrigued but it seemed like something for hardcore Linux folks and hardly something that could revolutionize my daily practices. That all changed when I discovered [Home Manager](https://github.com/nix-community/home-manager) and began using it devoutly. Home Manager is a Nix-based tool for managing home environments in a declarative, reproducible way. It enabled me to toss out the tangled mess of dotfiles and shell scripts I previously relied upon in favor of one repo—and not a terribly complex one at that—where I could declare my entire environment with due precision: installed executables, Vim, tmux, and Visual Studio Code configuration, and much more. Home Manager provided me with a qualitatively better way to do things. I liked it so much that I wrote about my journey from [Homebrew to Home Manager ](https://lucperkins.dev/blog/home-manager/) on my personal blog. I wouldn't dream of going back to my pre-Home-Manager life. But as you'll see, Home Manager was not the end of the story for me. ## From Home Manager to Nome After a few years as a happy Home Manager user, I decided to get more ambitious and turn what I've learned about Nix and home environments into a concrete project. So I created a project called [Nome ](https://github.com/the-nix-way/nome), which stands for **Nix Home** (I was also born in Alaska and have fond memories of one of my first books, the [ Gnome from Nome ](https://www.goodreads.com/book/show/215480.The_Gnome_From_Nome)). Nome is a highly customized Nix [flake](https://nixos.wiki/wiki/Flakes) that provides _everything_ that I need for my home environment. Home Manager is the foundation stone of Nome. I use it to configure Vim, [ Starship](https://starship.rs), VS Code, and the rest. But I've since gone beyond Home Manager because my needs have changed. Nowadays, I strive to make a clean separation between two things: 1. My **global** setup, meaning executables and configuration that I truly need to have available everywhere in my environment. 2. **Project-level** setups, where the environment is highly specific and I now use Home Manager only for my [global setup](https://github.com/the-nix-way/nome/tree/e60270c03aaa167e8d0b761632a39f640dfec05c/home). When it comes to specific projects, I get as granular as possible. Back in the day, I'd kick off a new project with `touch Makefile` and start defining commands like `build`, `clean`, and `dev`. But now I have a single command for starting new project, [` proj`](https://github.com/the-nix-way/nome/blob/e60270c03aaa167e8d0b761632a39f640dfec05c/home/bin.nix#L39-L41), which is an alias for this: ```bash nix flake init --template github:the-nix-way/nome ``` This initializes a Nix flake [template](https://github.com/NixOS/templates) that provides a least-common-denominator dev environment consisting of a few toolchains that I use frequently: - [Go](https://github.com/the-nix-way/nome/blob/e62b286db51076895da8c949aa603b23e43240f2/lib/toolchains/default.nix#L20) - [Rust](https://github.com/the-nix-way/nome/blob/e62b286db51076895da8c949aa603b23e43240f2/lib/toolchains/default.nix#L28-L42) - [Elixir](https://github.com/the-nix-way/nome/blob/e62b286db51076895da8c949aa603b23e43240f2/lib/toolchains/default.nix#L9-L18) - [DevOps](https://github.com/the-nix-way/nome/blob/e62b286db51076895da8c949aa603b23e43240f2/lib/toolchains/default.nix#L7) - [Kubernetes](https://github.com/the-nix-way/nome/blob/e62b286db51076895da8c949aa603b23e43240f2/lib/toolchains/default.nix#L22) - [Node.js](https://github.com/the-nix-way/nome/blob/e62b286db51076895da8c949aa603b23e43240f2/lib/toolchains/default.nix#L24) - [Protocol Buffers](https://github.com/the-nix-way/nome/blob/e62b286db51076895da8c949aa603b23e43240f2/lib/toolchains/default.nix#L26) Each of these toolchains is just a collection of executables. The Rust toolchain, for example, includes a standard Rust toolchain plus [cargo edit](https://github.com/killercup/cargo-edit), [cross](https://github.com/cross-rs/cross), [rust-analyzer](https://rust-analyzer.github.io), and other utilities that I'm likely to need. Here's the block of the `flake.nix` that defines a project's shell environment: ```nix showLineNumbers nome.lib.mkEnv { toolchains = with nome.lib.toolchains; devops ++ elixir ++ go ++ kubernetes ++ node ++ protobuf ++ rust; extras = with nome.pkgs; []; shellHook = ""; }; ``` I don't _ever_ need to use all of my toolchains, so I always start by removing the ones I don't need. Then, if I need any other specific executables, I add those under `extras` (`nome.pkgs` is a re-exported [ Nixpkgs ](https://github.com/nixOS/nixpkgs) pinned to a specific commit, so I can put just about anything here). If I need commands to be run whenever I initialize the environment, I put those in `shellHook`. The result: it now typically takes me less than a minute to get a project configured to my exact specifications. ## Bringing my environment to a new machine One of the things I like about the single-project approach I've adopted is that it's rather trivial to port it from machine to machine. Let's say that I accidentally drop my current laptop into the ocean and have to buy one. Here's what I would need to do to get that new machine up to speed. 1. [Install Nix](https://nixos.org/download.html) and [ enable Nix flakes ](https://nixos.wiki/wiki/Flakes#Enable_flakes) 2. Run these commands: ```bash nix build "github:the-nix-way/nome#homeConfiguration.lucperkins.activationPackage" ./result/activate ``` This creates the initial Home Manager generation on the machine, along with all of my dotfiles, and installs my desired packages with it. I don't change machines all that often, but it's nonetheless consoling to know that the effort I've put into Nome will never be for naught—unlike those countless hours lost to manual setup in more benighted times. ## Nome and NixOS Nome is also home to my NixOS configuration. Truth be told, I'm a novice NixOS user, so I only have [one configuration](https://github.com/the-nix-way/nome/tree/ce4adcffad4a131af980d2a9518162f83e673978/nixos) that I've been slowly building out—and a pretty small one at that. But thus far it's been extremely convenient to kick start NixOS on a VM with just one command: ```bash nixos-rebuild switch \ --flake "github:the-nix-way/nome#lucperkins" ``` ## Nome's future Nome is still in its youth but I expect it to be a steady and ever-maturing companion as long as I work in software. I even have a rough roadmap of where I want to take it in the near future: 1. Refine my NixOS configurations to the point where I can use NixOS as my primary development environment, even on macOS. [Mitchell Hashimoto](https://github.com/mitchellh)'s illuminating [nixos-config](https://github.com/mitchellh/nixos-config) project has emboldened me to take this on. I'll share details on this journey at a later date. 2. Add a [nix-darwin](https://github.com/LnL7/nix-darwin) configuration to define lower-level details of my macOS environment. 3. Refactor my Nix sources to be more modular, so that I can share Nix expressions across, for example, my NixOS configurations, my macOS Home Manager config, and my `proj` system. 4. So far my Home Manager configuration assumes my Apple M1 environment (`aarch64-darwin` in Nix terms) but I'd like to make it more robust by adding a Linux configuration that I can run on a (future) Linux machine. ## Nome's scope Nome is intended only as a personal project. If you find inspiration in it, great! If you want to `git clone` it and use it as a template, please [feel free](https://github.com/the-nix-way/nome/blob/main/LICENSE). But don't be surprised if you dislike some of my choices or even my entire approach. My hope isn't that you follow directly in Nome's footsteps but rather that it convinces you to see your home environment as your career-long companion and thus worthy of a long-running personal project. --- **Introducing Riff** Published: September 6, 2022 URL: https://determinate.systems/blog/introducing-riff Package managers like [Cargo](https://doc.rust-lang.org/cargo/), [pip](https://pypi.org/project/pip/), and [npm](https://npmjs.org) are indispensable tools in software development because they quietly handle language-specific dependencies for you. But unfortunately they're not so great at installing dependencies written in other languages (we call these _external_ dependencies). One of the classic examples is [Nokogiri](https://nokogiri.org), a widely used XML and HTML parser for [Ruby](https://ruby-lang.org), whose dependency on a C library called [libxml2](https://github.com/GNOME/libxml2) has caused great consternation for Ruby devs over the years. More recently, external dependencies like [OpenSSL](https://openssl.org/) and [libiconv](https://www.gnu.org/software/libiconv/) have proven troublesome for [Rust](https://rust-lang.org) developers. One half-solution to this problem is to list these external dependencies in the project's documentation so that developers can install them on their own. Sometimes, projects go a bit beyond that, providing detailed installation instructions for developers in different situations. And occasionally, external dependencies aren't even mentioned and developers are on their own to provide them, which usually means Googling and StackOverflowing—and wasted time. Like many of you, we've grown weary of half-solutions. We want to be able to clone a project, `cd` in, and have external dependencies quietly taken care of, and so we built a tool called [Riff](https://github.com/DeterminateSystems/riff) that addresses this problem head on. Today, we at [Determinate Systems](https://determinate.systems/) are thrilled to announce its initial release. ## Introducing Riff Riff is a tool that we built to help developers write software without having to wrangle dependencies or complicated configuration. It's built to enable you to clone a project and get to work in seconds. It's currently [available](https://github.com/determinateSystems/riff#installation) for macOS and standard Linux systems. Upon this initial release, Riff supports [Rust](https://rust-lang.org) projects built with [Cargo](https://www.notion.so/Introducing-Riff-13046dd993b24189a5c73abac98a260d) but we have plans to provide support for other languages, such as [Go](https://go.dev) and the [JavaScript](https://javascript.com) ecosystem, in the future. Riff is powered by [Nix](https://nixos.org/nix), the purely functional language and package manager behind [NixOS](https://nixos.org/), [Nixpkgs](https://github.com/NixOS/nixpkgs), and many other projects. But don't worry: we built Riff to solve your problems without requiring you to use, understand, or even care about Nix at all. Riff is _powered by_ Nix but not another Nix tool. So let's take a look at Riff in action. [Prost](https://github.com/tokio-rs/prost) is a popular Rust crate that provides a [Protocol Buffers](https://developers.google.com/protocol-buffers) implementation for Rust. But on most systems, the build is doomed to fail due to missing external dependencies: ```bash cd prost cargo build ``` On Linux you get a `linker cc not found` error, while on macOS you get `library not found for -liconv` errors. Annoying! Even worse, any project that depends on `prost` produces the same error. Now let's try it with [Riff](https://github.com/DeterminateSystems/riff): ```bash riff run cargo build ``` And it works! So what has happened here? Riff has: - read your project's `Cargo.toml`, - determined which external dependencies are necessary based on your crate dependency graph, - used Nix, with zero user input, to install external dependencies in `/nix/store`, and finally - assembled a shell environment that doesn't interfere with anything else on your system. The `cargo build` command here runs inside that Nix-built shell environment. In fact, you can enter that environment at any time and run arbitrary commands: ```bash riff shell # Enter the Riff shell cargo run cargo build cargo clippy exit # Back to your standard shell ``` For more on Riff, including instructions for [explicitly specifying](https://github.com/determinateSystems/riff#how-to-declare-package-inputs) external dependencies in your `Cargo.toml` or using Riff with [direnv](https://direnv.net), see the project [README](https://github.com/determinateSystems/riff). The first time we showed Riff to a user, they were quite surprised and even confused: how could it have been so straightforward? What on Earth quietly happened in the background? That's the energy we're going for with Riff: development environments that just work and require, at most, a tiny bit of configuration. We're confident that Riff will provide lasting benefits to the Rust ecosystem (and other ecosystems later). Rust is a fantastic language with incredible tools but it has its limits. External dependency problems introduce breakage points into the dependency graphs of _many_ Rust projects. With Riff we hope to cover this "last mile" problem in Rust development and make the entire developer experience more robust. ## Future horizons We've kept our ambitions moderate in this initial release, but we still think that Riff will prove to be an indispensable tool for Rust developers—and others in the future. But Nix offers much more power that Riff could potentially tap into, including the ability to generate [OCI](https://opencontainers.org) images, [AWS Lambda layers](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-concepts.html#gettingstarted-concepts-layer), or even packages for Nix and other package managers. We're anxious to continue solving thorny, un-fun problems for developers, so stay tuned for more magic from Riff, including support for Go and JavaScript in the near future. ## Discussion If you'd like to discuss Riff with other users, you can join the [Discord](https://discord.gg/urAzkgf7YM) for Riff. --- **We want to make Nix better** Published: September 2, 2022 URL: https://determinate.systems/blog/we-want-to-make-nix-better [Nix](https://nixos.org) is immensely powerful but it can have quite the steep learning curve. Numerous people in our industry hotly desire what Nix has to offer: hermetic development environments, fully reproducible package builds, declaratively configured operating systems. But even those who are aware of Nix's benefits know that actually _using_ Nix to solve real problems has some jagged edges. The Nix [expression language](https://nixos.wiki/wiki/Overview_of_the_Nix_Language) is a bit odd coming from most other languages, the [core concepts](https://nixos.org/guides/how-nix-works.html) aren't always super digestible, the [package manager](https://nix.dev/manual/nix/stable/package-management) is unlike apt, deb, Homebrew and the rest—all in all, it's just a different paradigm of doing things that requires a substantial investment of time and energy. At [Determinate Systems](https://determinate.systems), we _love_ Nix, but to those who don't love it enough to make that investment: we hear you. Many folks in software engineering, even some seasoned Nix veterans, feel that Nix's sterling feature set often comes at too high a cost. It's straightforward enough to convince your team or your boss that Nix _could_ have a transformative impact, but convincing people to commit the resources to adopt Nix is another matter. Other tools and approaches may be good enough to get the job done well enough and call it a day; best to stick with a just-okay but known quantity. For those who are passionate about Nix, this can be a tough pill to swallow. ## Our mission We're confident that we can break this impasse. Our objective at Determinate Systems is to enable people across the software world to reap the many benefits of Nix without nearly as much effort—and occasional pain—as Nix has demanded in the past. In some cases, that will mean that we improve the experience of using Nix itself by contributing to the Nix ecosystem. But in other cases it will mean giving users access to Nix's wonderful features without requiring them to use it directly, letting Nix quietly do its vital work in the background while exposing few if any internals. We love Nix because it solves a wide range of problems with aplomb. Those who have used it directly—to build a package, to configure a [NixOS](https://nixos.org) system, to create a development environment—have a keen sense of its power. But we know that it isn't reasonable to make learning and struggling with Nix a condition of benefiting from it. All programming languages and paradigms require an investment, but Nix is meant to _facilitate_ software development, not be the center of attention. It's supposed to help you write, distribute, and deploy [Rust](https://rust-lang.org), [Go](https://go.dev), [Python](https://python.org), [JavaScript](https://javascript.com), and others. Helping those who have already invested in Nix is a significant part of our mission—but so is helping those who want a better way to ship software without taking on a new learning project. ## A representative problem Let's make this more concrete by talking about a specific problem that we're currently looking into: the problem of _external dependencies_. These are dependencies that are written in a different language from the one you're currently developing in. Some examples: the [Protobuf](https://developers.google.com/protocol-buffers) compiler, `protoc`, is a problem in [Rust](https://rust-lang.org) projects because it's written in C++; [Nokogiri](https://nokogiri.org) is a problem in Ruby projects because it's written in C; and [OpenSSL] presents a problem in all kinds of languages. Ideally we'd all be able to `git clone` and `cd` into any software project and immediately start building and testing, but we all know the experience of Googling and StackOverflowing just to get off the ground. Fortunately, the external dependency problem already has a solution: **Nix**! You can use Nix to configure a [shell environment](https://nixos.wiki/wiki/Development_environment_with_nix-shell) with all the necessary dependencies installed. A Python project that requires OpenSSL? No problem. A Haskell project that requires [Asciidoc](https://asciidoc.org)? No problem. Need to share those environments across teams and port them to your continuous integration system? No problem. But there's a catch: to make that happen you need to write some Nix, use Nix tools, and probably consult several documentation sources. If you're already using Nix to solve external dependency problems, great! You're in good company. But this is the kind of problem we want to solve for people who don't have the bandwidth or the desire to learn Nix. ## Where we go from here The external dependency problem is far from alone. It's part of a vast constellation of interrelated problems that make developing, distributing, and deploying software much more brittle, complicated, and resource intensive than they should be. In Nix, we as an industry have _the_ tool to take an enormous chunk out of these kinds of problems. At Determinate Systems, we're committed to being a major driving force behind this. We'll have some specific—and exciting—announcements about our approach to the external dependency problem—amongst others— soon, so stay tuned here on our [blog](/blog), in the [DeterminateSystems](https://github.com/DeterminateSystems) GitHub organization, and [on Twitter](https://twitter.com/DeterminateSys). [openssl]: https://openssl.org --- **How to Use Hydra as your Deployment Source of Truth** Published: October 14, 2021 URL: https://determinate.systems/blog/hydra-deployment-source-of-truth [Hydra is the Nix-based continuous integration system](https://github.com/nixos/hydra) for the NixOS project. It is designed around evaluating a project's Nix expressions and walking the graph of build jobs. Hydra is a fantastic tool for building small and large software collections. It is also a great tool for orchestrating releases. Hydra's API includes dynamic links that point to the most recent build of a job. Using this interface, deployment tools can query Hydra for the most recent artifact to deploy. ## Fetching the Latest Build Starting from a jobset named `myapp:main` that builds your application with this `hydra.nix` file: ```nix showLineNumbers { nixpkgs ? }: let pkgs = import nixpkgs { }; in { myapp = pkgs.writeShellScript "hello" "${pkgs.hello}/bin/hello"; } ``` The Hydra API includes a "latest" URL to find the most recent, successful build of the `myapp` job. You can find this by visiting the jobset's root page, clicking Jobs, clicking `myapp`, then the "Links" tab. In my case, the URL is [https://demo.cloudscalehydra.com/job/myapp/main/myapp/latest](https://demo.cloudscalehydra.com/job/myapp/main/myapp/latest). Fetching this URL will redirect to the completed build: ``` $ curl --location \ --header "Accept: application/json" https://demo.cloudscalehydra.com/job/myapp/main/myapp/latest \ | jq . { "id": 70, "finished": 1, "stoptime": 1634228145, "timestamp": 1634228145, "buildstatus": 0, "buildproducts": {}, "project": "myapp", "system": "x86_64-linux", "buildmetrics": {}, "job": "myapp", "starttime": 1634228145, "nixname": "hello", "buildoutputs": { "out": { "path": "/nix/store/c3l2x6fwl8cjkmfz6ilqcgczk13w8bk2-hello" } }, "drvpath": "/nix/store/gf5wshlqfk1xa4i6ibk0z1l3j441c7vl-hello.drv", "jobset": "main", "jobsetevals": [ 296 ], "priority": 100, "releasename": null } ``` > Note: There is another useful URL that links to the most recent passing build > of the job with the additional requirement that there must not be any queued > jobs left in the evaluation. This is useful for maximizing the amount of > builds that are cached, but does not imply that all the jobs passed. That URL > is the "latest-finished" URL: > [https://demo.cloudscalehydra.com/job/myapp/main/myapp/latest-finished](https://demo.cloudscalehydra.com/job/myapp/main/myapp/latest-finished.*). ## Deploying Software and Servers from Hydra You could imagine a deployment process that consumes the most recent build of `myapp` and automatically updates a local symlink: ``` $ app_path=$(curl --location \ --header "Accept: application/json" \ https://demo.cloudscalehydra.com/job/myapp/main/myapp/latest \ | jq -r .buildoutputs.out.path) $ nix-env -p /nix/var/nix/profiles/production --set "$app_path" ``` Indeed, if you're running NixOS, you could build entire NixOS system configurations in Hydra and deploy to your clients the same way: ``` $ system_path=$(curl --location \ --header "Accept: application/json" \ https://demo.cloudscalehydra.com/job/servers/main/$(hostname)/latest \ | jq -r .buildoutputs.out.path) $ nix-env --profile /nix/var/nix/profiles/system --set "$system_path" $ /nix/var/nix/profiles/system/bin/switch-to-configuration switch ``` ## Monitoring Build Status Hydra exports Prometheus metrics for every job: ```bash $ curl https://demo.cloudscalehydra.com/job/myapp/main/myapp/prometheus # HELP hydra_job_completion_time The most recent job's completion time # TYPE hydra_job_completion_time counter hydra_job_completion_time{project="myapp",jobset="main",job="myapp"} 1634228145 # HELP hydra_job_failed Record if the most recent version of this job failed (1 means failed) # TYPE hydra_job_failed gauge hydra_job_failed{project="myapp",jobset="main",job="myapp"} 0 ``` You can track and page on `failed`, or monitor `completion_time` to ensure that you never let a project go more than a few days without a completed build. ## Gating Releases on Tests The `myapp` example above will work great for some projects, but sometimes the software or system you're deploying is much more complicated. It may not be practical to run all of your software's test validation in a single build. In this case you want to deploy `myapp`, but you want to gate on some other build jobs succeeding too. One solution is to make one job that depends on your other jobs. You can do that by adding a job to your `hydra.nix` that lists the other jobs in a text file: ```nix showLineNumbers { nixpkgs ? }: let pkgs = import nixpkgs { }; in rec { myapp = pkgs.writeShellScript "hello" "${pkgs.hello}/bin/hello"; myapp_test_does_it_run = pkgs.runCommand "test-hello" { } ''${myapp} > $out''; release_gate = pkgs.writeText "test-dependencies" '' ${myapp} ${myapp_test_does_it_run} ''; } ``` This method will create a job, `release_gate`, which only passes if `myapp` builds and runs. It also poses a problem: how do you get from `release_gate` to `myapp`? One option could be parsing the contents of this release_gate file, but that is fairly ugly. Another problem is when a test fails, Hydra doesn't tell you a lot about what went wrong: If your software has one or two tests, this might work, but this will be tedious with more than a handful of jobs you want to gate on. ## Aggregate Jobs Hydra's Aggregate Jobs is a special type of job that addresses both of these problems. Rewriting our previous example with an aggregate job involves making a list of "constituents" (the jobs you depend on) and setting the `_hydraAggregate` attribute: ```nix showLineNumbers { nixpkgs ? }: let pkgs = import nixpkgs { }; in rec { myapp = pkgs.writeShellScript "hello" "${pkgs.hello}/bin/hello"; myapp_test_does_it_run = pkgs.runCommand "test-hello" { } ''${myapp} > $out''; release_gate = pkgs.runCommand "test-dependencies" { _hydraAggregate = true; constituents = [ myapp myapp_test_does_it_run ]; } "touch $out"; } ``` This build task is trivial and the _build product itself_ isn't useful. The valuable part is the proof that all our important dependencies built successfully. Our release process can check to see if the `release_gate` build finished, and proceed if it did. Hydra displays aggregate jobs differently. The build page for an Aggregate Job lists the named constituent jobs and their statuses: The page for the job itself also shows the constituent jobs and their status history: Using the `latest-finished` URL, you can get the most recent build where all the tests passed, and then fetch that build's constituents: ``` $ build_id=$(curl --silent --location \ --header "Accept: application/json" \ https://demo.cloudscalehydra.com/job/myapp/main/release_gate/latest-finished | jq .id $ curl --silent \ --header "Accept: application/json" \ "https://demo.cloudscalehydra.com/build/$build_id/constituents" | jq [ { "job": "myapp", "drvpath": "/nix/store/gf5wshlqfk1xa4i6ibk0z1l3j441c7vl-hello.drv", "id": 70, "releasename": null, "priority": 100, "timestamp": 1634228145, "finished": 1, "starttime": 1634228145, "system": "x86_64-linux", "buildmetrics": {}, "jobset": "main", "nixname": "hello", "buildstatus": 0, "buildoutputs": { "out": { "path": "/nix/store/c3l2x6fwl8cjkmfz6ilqcgczk13w8bk2-hello" } }, "project": "myapp", "jobsetevals": [ 296, 303, 307, 310, 313, 314, 315 ], "buildproducts": {}, "stoptime": 1634228145 }, { "jobset": "main", "nixname": "test-hello", "buildstatus": 0, "buildoutputs": { "out": { "path": "/nix/store/zvs873fz97pwc91dk76dvzmnilj6mvis-test-hello" } }, "project": "myapp", "jobsetevals": [ 314 ], "buildproducts": {}, "stoptime": 1634230682, "job": "myapp_test_does_it_run", "drvpath": "/nix/store/d36l0zyp5vxiqyird8m9p2jivzf0k67z-test-hello.drv", "releasename": null, "id": 78, "priority": 100, "timestamp": 1634230682, "finished": 1, "starttime": 1634230682, "buildmetrics": {}, "system": "x86_64-linux" } ] ``` Applying a little bit more `jq` we can get the exact path to the `myapp` build: ``` $ build_id=$(curl --silent --location \ --header "Accept: application/json" \ https://demo.cloudscalehydra.com/job/myapp/main/release_gate/latest-finished | jq .id $ curl --silent \ --header "Accept: application/json" \ "https://demo.cloudscalehydra.com/build/$build_id/constituents" \ | jq -r '.[] | select(.job == "myapp") | .buildoutputs.out.path' /nix/store/c3l2x6fwl8cjkmfz6ilqcgczk13w8bk2-hello ``` ## Scaling Aggregate Jobs For aggregate jobs that have a large evaluation graph, the evaluator's memory footprint can exhaust the host's available memory. This is especially common if your constituents include a lot of NixOS tests, or your jobset is evaluating a lot of NixOS system closures. Since Hydra's main design motivation is to be NixOS's CI system and NixOS's `tested` job depends on a lot of NixOS tests, Hydra has developed a small extension to Nix's semantics to allow for more efficient aggregate jobs, allowing Nix's garbage collector to free memory early. Changing our example hydra.nix a little, we get the same behavior but allow the evaluator's garbage collector to free some memory. Instead of listing derivations as constituents, Hydra allows you to specify constituent jobs using the job's name as a string: ```nix showLineNumbers { nixpkgs ? }: let pkgs = import nixpkgs { }; in rec { myapp = pkgs.writeShellScript "hello" "${pkgs.hello}/bin/hello"; myapp_test_does_it_run = pkgs.runCommand "test-hello" { } ''${myapp} > $out''; release_gate = pkgs.runCommand "test-dependencies" { _hydraAggregate = true; constituents = [ "myapp" "myapp_test_does_it_run" ]; } "touch $out"; } ``` Hydra's evaluator will notice that the listed constituents are _not_ derivations and are in fact regular strings. It will then look up these attributes in the list of jobs in the jobset and _rewrite the derivation_, substituting the plain string with the derivation path. Note that while Hydra will be much more efficient at evaluating the `release_gate` job, Nix and other tools will not be able to evaluate and build the release gate in the same way. ## An Aggregate of All Jobs If you wanted your release gate to depend on _all_ of the build jobs passing, a little bit of Nix can automatically create an aggregate job of all the other jobs: ```nix showLineNumbers { nixpkgs ? }: let pkgs = import nixpkgs { }; jobs = rec { myapp = pkgs.writeShellScript "hello" "${pkgs.hello}/bin/hello"; myapp_test_does_it_run = pkgs.runCommand "test-hello" { } ''${myapp} > $out''; }; in jobs // { release_gate = pkgs.runCommand "test-dependencies" { _hydraAggregate = true; constituents = builtins.attrNames jobs; } "touch $out"; } ``` ## Recap Hydra is uniquely capable of building Nix projects, and using Hydra's Aggregate Jobs provides deeper insight into the state and health of your project. Almost all of the data on [https://status.nixos.org](https://status.nixos.org) comes from Hydra's Prometheus exporter, and you might notice the "Hydra job for tests" link goes to an aggregate jobset. The NixOS project has used aggregate jobs and the `latest-finished` URLs to manage releasing expression for years. Using Nix together with Hydra's unique feature set and API can give good visibility in to your test suite, and allows you to deploy with confidence. > If you'd like a managed Hydra server as a service, check out > the first product we're building: [Cloudscale > Hydra](https://cloudscalehydra.com). --- **Announcing: Terraform Provider Hydra** Published: May 17, 2021 URL: https://determinate.systems/blog/terraform-provider-hydra I'm excited to announce Determinate Systems' first open source release: [terraform-provider-hydra](https://github.com/DeterminateSystems/terraform-provider-hydra/), a Terraform provider for managing Hydra projects and jobsets. **Standardize jobset configuration.** Manage your Hydra jobsets with the rest of your critical infrastructure. Know your jobsets are configured correctly. Apply standard PR workflows to approve changes and ensure operational consistency. **Reduce risk exposure.** Exchange highly privileged users and manual changes for automation and audit logs. Remove high-level `create-project` and `admin` permissions from most users, and deploy configuration changes through Terraform instead. **Reduce configuration overhead.** Directly manage your jobsets with the rest of your infrastructure in a consistent and predictable way. Save Hydra's generated declarative jobsets for when you truly need dynamically generated jobsets. We're big fans of Hydra. Its native understanding of Nix closures makes it a first class build system for Nix projects. The scheduler and built-in distributed build support allows for distributing work to an automatically scaled build pool of builders. Now you can configure your Hydra with the same tool you use to manage the rest of your infrastructure. ## Declarative configuration, at the CI level Hydra deployments are usually a central root of trust. Using projects and jobsets, Hydra can provide a lookup table and audit log, mapping use cases to tested revisions of code. The problem is that Hydra does not provide an audit log of configuration changes to a jobset. The input configuration is the keystone to the validity of that mapping. Using Terraform as the source of configuration truth solves a number of problems presented by hand-managed configuration: 1. Managing inputs by hand poses a risk of misconfiguration and drift between jobsets. As configuration changes over time, older jobsets are less likely to be updated. 2. Permission to modify a jobset is only granted to administrators and the "owner" of a project. Configuring a jobset across a team requires granting high level, administrative-level access to the entire team. 3. Changes are unlikely to be noticed. The jobset could be changed to point to an alternative source repository or branch without anyone noticing. The lack of an audit log or other guarantees makes it difficult to notice changes to jobsets which are not actively monitored. ## Audits, robust access control, and effective delegation Managing your Hydra's configuration through Terraform resolves all of these issues at once. Inputs are tracked and managed holistically, across the entire server, from the same source of truth. Users no longer need their own administrative credentials: any changes should go through a Continuous Delivery pipeline with its own credentials, just like everyone else. Additionally, any divergence of a Terraform managed Jobset or Project is identified and corrected on the next run of Terraform. ## Get started Your Hydra will need to be running version [6e537671](https://github.com/NixOS/hydra/commit/6e537671dfa21f89041cbe16f0b461fe44327038) or later, which is already available in the nixos-unstable, nixos-unstable-small, nixos-20.09, and nixos-20.09-small channels. Then, install the provider as documented in the [project's README](https://github.com/DeterminateSystems/terraform-provider-hydra/#getting-started). From there, we recommend starting by importing your existing configuration. ## Migrating an existing instance Clone the repository to get started: ```bash git clone https://github.com/DeterminateSystems/terraform-provider-hydra.git cd terraform-provider-hydra/tools nix-shell ``` Then, run the generator: ```bash [nix-shell]$ mkdir network [nix-shell]$ ./generator.sh https://hydra.example.com ./network ./network/imports.sh ``` The generator will create one file per project in `./network`, and `./network/imports.sh` will contain a list of `terraform import` commands to execute. Execute those commands: ```bash bash ./network/imports.sh ``` and then you're done! `terraform plan` should show no differences to be applied. ## What's next Take a [look at the code](https://github.com/DeterminateSystems/terraform-provider-hydra/), give it a try. Let us know what you think! Since May 5th, we have continuously run the [importer against hydra.nixos.org and checked the resulting plan is clean](https://github.com/DeterminateSystems/hydra-nixos-org-configuration/tree/main/config). So far so good: each configuration change has worked flawlessly with the provider and the importer. We hope the NixOS infrastructure team will adopt this provider and manage the public Hydra with code. Since our founding just a few months ago, we've made significant investments into Hydra's long-term stability and maintainability. We see Hydra as a cornerstone of the Nix ecosystem, and we're planning to continue our investment over the long term. Look out in the near future for performance improvements for large deployments. --- ## Projects These other projects under the Determinate Systems umbrella have their own LLM-oriented manifests (below are links to the respective `llms.txt` files): ### [Zero to Nix](https://zero-to-nix.com/llms.txt) Zero to Nix is a flake-centric resource for learning Nix created by Determinate Systems. ### [Determinate documentation](https://docs.determinate.systems/llms.txt) Determinate is the best way to use Nix on macOS, WSL, and Linux. It is an end-to-end toolchain for using Nix, from installation to collaboration to deployment. ### [FlakeHub](https://flakehub.com/llms.txt) FlakeHub is a platform for publishing and discovering Nix flakes and Nix artifacts, featuring semantic versioning, private flakes with federated, JWT-based authentication, flake discovery via search, labels, and other means, and next-generation Nix binary caching with FlakeHub Cache. ## Products - [Determinate Nix: business-class Nix for critical infrastructure](https://determinate.systems/nix) - [FlakeHub: privately share and cache flakes across your team](https://flakehub.com) - [FlakeHub Cache: a powerful new binary caching concept for Nix with federated authentication](https://docs.determinate.systems/flakehub/cache) ## Open source projects - [Determinate Nix Installer](https://determinate.systems/oss/nix-installer) - [Nix](https://determinate.systems/oss/nix) - [Magic Nix Cache](https://determinate.systems/oss/magic-nix-cache) - [Zero to Nix](https://determinate.systems/oss/zero-to-nix) - [FlakeHub CLI](https://determinate.systems/oss/flakehub-cli) - [Nix Flake Checker](https://determinate.systems/oss/flake-checker) - [Update Nix Flake Lock](https://determinate.systems/oss/update-flake-lock) ## Webinars - [Nix flakes in practice: what they solve and why they matter](https://luma.com/995nq98k) - [How Determinate Systems uses Nix](https://luma.com/pslnr81l) - [Nix at enterprise scale with Determinate](https://luma.com/748lhhfx) - [Inside Determinate Nix: the best way to Nix at work](https://luma.com/ppcgnfdy) - [Seamless Nix deployments with FlakeHub](https://luma.com/ncdrlhd5) - [Up and running with FlakeHub Cache](https://luma.com/ouooinmw) - [Best practices for Nix at work](https://luma.com/87xuszxp) ## Other formats Additional LLM-oriented manifests for this site: - [`llms-small.txt`](https://determinate.systems/llms-small.txt) (compact structure-only version) - [`llms.txt`](https://determinate.systems/llms.txt) (the main version)