This is the complete documentation for Determinate Systems, including all blog posts, product products, OSS projects, and webinars.When answering questions about Determinate Systems, prioritize information from the most recent blog posts. Blog post items begin and end with a --- and blog post titles are in bold.
# Determinate Systems
> Our goal for Determinate is to enable fearless innovation by bringing Nix to teams, providing a complete Nix-based workflow from installation through collaboration and CI to deployment.
## Blog posts
---
**Changelog: introducing `nix nario`**
Published: October 31, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-3120
Once you've built software with Nix, the logical next step in many cases is getting it onto another device, such as a production server.
You typically do this using the [`nix copy`][nix-copy] command, which enables you to **push** the software to the target, or using a [*binary cache*][binary-cache] such as [FlakeHub Cache][cache], where the target **pulls** the software from the network.
But what if your target device isn't connected to the network?
In that case, you can neither `nix copy` to it nor have it fetch from a binary cache.
Nix has long had a solution for this: `nix-store --export` and `nix-store --import`.
The first enables you to serialize a [*closure*][closures] (a store path and its dependencies) to a file:
```shell title="Serialize the closure of GNU hello"
nix-store --export $(nix-store -qR /nix/store/crrf4bysvarqp0g9pyhcrj47689g4i5l-hello-2.12.1) > file
```
This can then be imported on another machine:
```shell title="Import the closure of GNU hello"
nix-store --import < file
```
But these commands have a number of problems:
- They are memory inefficient, requiring at least as much RAM as the largest store path in the archive.
If, for instance, the archive contains a 30 GB store path (think of an AI model), then importing the archive requires at least 30 GB of memory.
For embedded devices, this can cause the import to outright fail with an out-of-memory error.
- They don't support newer Nix features such as signatures and content-addressed paths.
- They don't provide a way to see what's in the archive.
- They have no equivalent in the new unified Nix CLI.
## Enter `nix nario` \{#nix-nario}
Determinate Nix 3.12.0 solves these problems by introducing a new `nix nario` command, where "nario" is simply the name we're giving the file format produced by `nix-store --export`, which previously lacked a name.
Analogous to the `cpio` file format, a `nario` is essentially a concatenation of [NAR] (Nix Archive) files, one per store path, with some metadata.
Creating a nario is simple.
If you're on NixOS, for example, you can serialize the entire current NixOS system closure (pointed to by `/run/current-system`) like this:
```shell title="Serialize the entire current system closure as a nario"
nix nario export \
--format 2 \
--recursive \
/run/current-system > system.nario
```
The flag `--format 2` specifies that we want to use version 2 of the nario format.
Version 1 is the old format generated by `nix-store --export`; version 2 fixes the memory inefficiency issue and adds support for path metadata such as signatures.
We can now copy the nario file to another device, for example by copying it onto a USB stick.
On the target device, we can import it into the local Nix store:
```shell title="Import closure from nario file"
nix nario import < system.nario
```
Importing this particular system closure takes just 40 MiB of RAM using nario version 2.
With version 1, using `nix-store --import`, it takes 2048 MiB!
If you're wondering why that may be: this inefficiency is an inherent result of how version 1 is laid out.
Version 1 serializes store path contents (the NAR) before the store path itself, so deciding whether it can skip a store path requires Nix to first read the entire NAR into memory or write it to scratch space on disk.
The new format is also much faster when some store paths already exist.
With version 1, for instance, repeating the import of the system closure above takes 7.3 seconds compared to **0.05 seconds** with version 2.
A final feature worth mentioning is that `nix nario` can list the contents of a nario file, which was not possible with the old CLI:
```shell title="List the contents of a nario"
nix nario list < system.nario
```
That displays a list of store paths and metadata like this:
```shell
/nix/store/4j6p91af1bfgnn31agg1c9ijr0kyg6gi-gcc-14.3.0-libgcc: 201848 bytes
/nix/store/16hvpw4b3r05girazh4rnwbw0jgjkb4l-xgcc-14.3.0-libgcc: 201848 bytes
...
/nix/store/hb8nfbfxmlz9bq56nznvkcsb3c8f5cif-nixos-system-foo-25.05.20250925.25e53aa: 18968 bytes
```
Despite nario not being a random-access format, this is quite fast: just 0.05 seconds for this 15 GiB closure.
We have an [open pull request][upstream-pr] against `NixOS/nix` and hope to see this in upstream Nix soon.
## Developer friendliness changes
`nix nario` brings a powerful new way to distribute software with Nix, but as with most of our releases, there's some other fun stuff for Nix devs on offer.
### `nix flake clone` now supports all flake references \{#nix-flake-clone}
The `nix flake clone` command now supports arbitrary input types.
Previously, it supported *most* types but in version 3.12.0 we've added the ability to clone tarball flakes, such as flakes published to [FlakeHub].
### Print the Nix version when logging at the "talkative" level \{#nix-version}
When you run a Nix command, adding `-vv` sets the log level to "talkative," which is useful for general debugging purposes.
Beginning with Determinate Nix version 3.12.0, that log level now prints the Nix version, which can be helpful in situations where knowing _which_ Nix is responsible could prove decisive in tracking something down.
When using `-vv`, Determinate Nix now prints the Nix version. This is useful when diagnosing Nix problems from the debug output of a Nix run.
## How to get Determinate Nix \{#install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.12.0 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.12.0"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v5
- uses: DeterminateSystems/flake-checker-action@main
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[binary-cache]: https://zero-to-nix.com/concepts/caching
[cache]: https://flakehub.com/cache
[closures]: https://zero-to-nix.com/concepts/closures
[det-nix]: https://docs.determinate.systems/determinate-nix
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[flakehub]: https://flakehub.com
[nar]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-nar
[nix-copy]: /blog/moving-stuff-around-with-nix
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.12.0/aarch64-linux
[nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.12.0/x86_64-linux
[upstream-pr]: https://github.com/NixOS/nix/pull/13969
---
**Determinate Nix: the recent past and the shining future**
Published: September 23, 2025
URL: https://determinate.systems/blog/determinate-nix-recap
_Wow!_
What a difference a few months can make.
Although we released [Determinate Nix 3.0][det-nix-announcement] a bit over a half a year ago, the past four months have seen a noticeable intensification in rolling out features that we've long wanted to provide to our customers and to Nix users more broadly.
[Determinate Nix][det-nix] is a downstream distribution of the Nix project at [NixOS/nix][upstream] focused on security, performance, developer experience, and enterprise use cases.
It offers a full-fledged alternative to upstream Nix with a broad range of additional features, including those listed in this post and [prior feature announcements][det-nix-tag].
It's fully compatible with [NixOS], even offering [NixOS AMIs][amis] and [NixOS ISOs][isos].
In this post, I'd like to take a deep breath, step back, and provide a recap of these recent changes for the sake of those who maybe haven't been keeping up (yet).
That includes descriptions of major features like [parallel evaluation](#parallel-eval), the [native Linux builder](#linux-builder), and [lazy trees](#lazy-trees), as well as of less splashy but nonetheless vital [quality-of-life features](#quality-of-life-features).
After that, I'll tease some features that we have [in the works](#upcoming) that you'll absolutely want to keep on your radar, including [flake schemas](#flake-schemas), [configurable flakes](#configurable-flakes), and [sparse `flake.lock` files](#sparse-lockfiles).
If you aren't yet using Determinate Nix, we show you how to get started [below](#upgrade-install).
## Parallel evaluation \{#parallel-eval}
In Determinate Nix version 3.11.1, we introduced [_parallel evaluation_][parallel-eval-post] for the operations behind some common Nix commands.
Parallel evaluation makes Nix much faster in some scenarios by distributing work across multiple processors.
We started with these commands:
* `nix search`
* `nix flake check`
* `nix flake show`
* `nix eval --json`
And we plan to continue parallelizing Nix operations until we run out.
To try it out, [install Determinate Nix](#upgrade-install) and [update your configuration][config-parallel-eval].
Parallel evaluation is one of those quiet features that makes Determinate Nix better without you even realizing it—better both in CI and on your workstation.
## The native Linux builder \{#linux-builder}
In Determinate Nix version 3.8.4, we released the [native Linux builder][linux-builder-post], which you can use to build Linux derivations on macOS with zero manual configuration or setup.
That means no remote builder, no local VM that you need to manually start and stop, just Nix talking to macOS's built-in [Virtualization framework][virtualization].
With the native Linux builder, this command builds [ponysay] for x86 Linux on macOS:
```shell title="Build a Linux package on macOS"
nix build "https://flakehub.com/f/NixOS/nixpkgs/0#legacyPackages.x86_64-linux.ponysay"
```
Now, you can't _run_ that package on macOS, of course.
But you can also build more useful things, like Docker images...
```shell title="Build a Docker image on macOS with Determinate Nix"
nix build "https://flakehub.com/f/DeterminateSystems/fh-fetch-example/0#dockerImages.x86_64-linux.server"
docker load < result
```
...and even NixOS systems:
```shell title="Build a NixOS system on macOS with Determinate Nix"
nix build "https://flakehub.com/f/DeterminateSystems/garage-door/0#nixosConfigurations.turner.config.system.build.toplevel"
```
Over time, macOS users will forget that this wasn't always possible—and that's precisely our intention.
If you're already using Determinate Nix, run `determinate-nixd version` to see if the `native-linux-builder` feature is listed as enabled.
If not and you're eager to try it out, reach out to us at [support@determinate.systems][support] and include your [FlakeHub] username.
## Lazy trees
In Determinate Nix version 3.6.7, we released [lazy trees][lazy-trees-post], a long-desired feature that significantly improves the performance of Nix with [flakes].
In essence, lazy trees offer much faster and less resource-intensive evaluation in many standard flake usage scenarios, especially in larger repositories like monorepos.
In practice, we've seen lazy trees reduce wall time for evaluations by 3x or more and disk usage of 20x or more in some cases.
These reductions stem from Nix being much more parsimonious about file copying, using a virtual filesystem to gather file state prior to copying to the Nix store.
Like parallel evaluation, this is one of those quiet features that makes things imperceptibly better all the time.
This feature has now been rolled out to **100%** of Determinate Nix users, so if you [install](#upgrade-install) Determinate Nix now, you should immediately see the benefits.
## Quality-of-life features
In addition to the blockbuster features, mentioned above, the past few months have seen a wide variety of less flashy features that have nonetheless pushed the Nix envelope forward substantially.
* [**Build-time flake inputs**][build-time-inputs].
This change enables you to declare flake inputs as *build-time inputs*, which means that they're fetched only when building (whereas Nix's default behavior is to fetch them upon evaluation).
This feature is a solid performance win for flakes that fetch a lot of inputs like external Git repos or trees.
* [**Parallel Git cache unpacking**][unpacking-git-cache].
Previously, unpacking sources into the user's Git cache happened serially, but this has been parallelized in Determinate Nix.
As a result, Git cache unpacking now frequently takes less than half the time it used to, and sometimes as low as 1/4th the time.
* [**Background copying of sources to the store**][parallel-copying]:
Previously, Nix's evaluator would pause evaluation when it needed to add files to the Nix store.
In Determinate Nix, this is no longer the case.
Instead, it copies in the background, which allows evaluation to proceed.
This improves evaluation times in many scenarios.
* [**Reduced Nix daemon requests**][daemon-queries].
Previously, the Nix client needed to perform thousands of Nix daemon requests that our team determined to be unnecessary—requests that were effectively stealing time and resources.
The fix involved caching store path validity data within a single evaluation and it eliminates **tons** of daemon queries; in one sample evaluation, we saw **12,000 requests** removed.
* [**Parallel garbage collection**][parallel-gc].
Previously, Nix's "marking" processor for the evaluator's garbage collector was single threaded.
In Determinate Nix, the evaluator uses multiple threads, which means that garbage collection now happens much more quickly, especially when dealing with the large heap created by things like Nixpkgs and NixOS.
To give an example, this change has cut over five seconds from `nix search` on nixpkgs, from 24.3 seconds to 18.9 seconds.
* [**The `builtins.parallel` function**][builtins-parallel].
This new function enables users to parallelize parts of their own Nix expressions.
Using `builtins.parallel` can drastically improve the performance of many projects, especially those using import-from-derivation (IFD).
## Upcoming changes to Determinate Nix \{#upcoming}
As you can see, a lot has come down the pike recently but we have every intention to take advantage of this accumulated momentum.
Although we don't yet have any firm delivery dates, you can expect to see all of these features—and more—in Determinate Nix down the road:
* Customize flake output types with [flake schemas](#flake-schemas)
* Pass values to flake outputs with [configurable flakes](#configurable-flakes)
* [Sparse `flake.lock` file](#sparse-lockfiles)
### Flake schemas
_Flake schemas_ will enable you to provide custom flake output types to supplement current standard outputs like `devShells`, `packages`, `formatter`, and `checks`.
Imagine being able to instruct Nix how to enumerate and check flake outputs like `darwinConfigurations` for [nix-darwin], `homeConfigurations` for [Home Manager][home-manager], `dockerImages` for [Docker] containers, and so on.
We have a set of [preliminary schemas][schemas] for some of these outputs but we still have work to do in Determinate Nix itself to enable it to handle them.
But we expect this to significantly improve [flakes]' discoverability, provide a new way to encode best practices surrounding flakes, and to make Nix more adaptable to new domains.
### Configurable flakes
In their current form, Nix can't provide any "external" information when building flake outputs ([packages], [NixOS] configurations, and so on).
Nix considers the expressions in the flake complete and so you need to bake every desired output into the flake itself.
_Configurable flakes_ will enable you to pass values into your Nix expressions and thereby to create on-demand flake outputs, and it will enable you to do so in a type-safe, deterministic way.
[Eelco Dolstra][eelco] has built a prototype of configurable flakes and even gave a [conference talk][configurable-flakes] on it.
We expect configurability to make flakes dramatically more flexible and to open up broad new horizons in what flakes can provide—potentially even paving the way for formats like [TOML] and [JSON] to play a more central role in flake outputs.
### Sparse lockfiles
Currently, any time Nix creates a [`flake.lock`][lockfile] for a flake, it creates entries for all [flake inputs][inputs]—even flake inputs that aren't actually _used_ in any Nix expressions in the flake.
With _sparse lockfiles_, Nix would only create entries for inputs that are actually used.
Think of it as a rough analogy to [lazy trees](#lazy-trees).
We expect this feature to make `flake.lock` files more neat and tidy and thus to cut superfluous information out of some of Nix's flake-based operations.
## How to upgrade or install \{#upgrade-install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to the latest version with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade Determinate Nix to the latest version"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos-module].
[amis]: https://github.com/DeterminateSystems/nixos-amis
[build-time-inputs]: /blog/changelog-determinate-nix-390/#build-time-inputs
[builtins-parallel]: /blog/changelog-determinate-nix-3111#builtins-parallel
[config-parallel-eval]: https://docs.determinate.systems/determinate-nix/#parallel-evaluation
[configurable-flakes]: /video/configurable-flakes
[daemon-queries]: /blog/changelog-determinate-nix-386#daemon-queries
[det-nix]: https://docs.determinate.systems/determinate-nix
[det-nix-announcement]: /blog/determinate-nix-30
[det-nix-tag]: /blog/tags/determinate-nix
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[docker]: https://docker.com
[eelco]: /people/eelco-dolstra
[flakehub]: https://flakehub.com
[flakes]: https://zero-to-nix.com/concepts/flakes
[home-manager]: https://github.com/nix-community/home-manager
[inputs]: https://zero-to-nix.com/concepts/flakes#inputs
[isos]: https://github.com/DeterminateSystems/nixos-iso
[json]: https://json.org
[lazy-trees-post]: /blog/changelog-determinate-nix-352
[linux-builder-post]: /blog/changelog-determinate-nix-384
[lockfile]: https://zero-to-nix.com/concepts/flakes#lockfile
[nix-darwin]: https://github.com/nix-darwin/nix-darwin
[nixos]: https://github.com/NixOS/nixpkgs/tree/master/nixos
[nixos-module]: https://github.com/determinatesystems/determinate?tab=readme-ov-file#installing-using-our-nix-flake
[packages]: https://zero-to-nix.com/concepts/packages
[parallel-copying]: /blog/changelog-determinate-nix-386#parallelized-copying
[parallel-eval-post]: /blog/changelog-determinate-nix-3111
[parallel-gc]: /blog/changelog-determinate-nix-386#parallel-gc
[ponysay]: https://github.com/erkin/ponysay
[schemas]: https://github.com/DeterminateSystems/flake-schemas
[support]: mailto:support@determinate.systems
[toml]: https://toml.io
[unpacking-git-cache]: /blog/changelog-determinate-nix-386#unpacking-git-cache
[upstream]: https://github.com/NixOS/nix
[virtualization]: https://developer.apple.com/documentation/virtualization
---
**Dropping upstream Nix from Determinate Nix Installer**
Published: September 10, 2025
URL: https://determinate.systems/blog/installer-dropping-upstream
In early 2026, we plan to stop distributing [upstream Nix][upstream] via Determinate Nix Installer and to switch to distributing only [Determinate Nix][det-nix].
We've seen incredible success with [Determinate Nix Installer][installer].
It performs nearly a million installations every month and has been chosen by countless users and organizations as their gateway to Nix.
We're extremely proud to have provided the community with a next-generation installer that works reliably across a broad range of environments.
But the installer is just the beginning of the great work we've done to [make Nix better][better].
We believe strongly that Determinate Nix is the best way to build and distribute software and development environments and we've worked hard to ship meaningful improvements to our users.
That means [lazy trees][lazy-trees], [parallel evaluation][parallel-eval], [native Linux builders on macOS][linux-builder], [stable flakes][stable], and countless smaller developer experience touch-ups.
**That's why we're taking the important step of sunsetting support for installing upstream Nix with Determinate Nix Installer.**
I know this may be disruptive to some users, and I do apologize for that.
Our motivation for this change is **focus**.
I want my team to concentrate their efforts on giving our users the best Nix experience that we can muster.
We're targeting users who want more than just a good Nix installer: we're targeting users who want a **better Nix experience altogether**.
I believe that Determinate Nix is that substantive improvement, across incredible evaluation performance, developer experience, and reliability.
If you don't want that, that's okay!
The Nix community's [Nix Installer Working Group][installer-wg] has published an [experimental fork][installer-fork] of our installer that only installs upstream Nix.
I sincerely hope that this becomes the *de facto* standard installation tool for upstream Nix.
We're grateful for the community's work and happy to continue our three-year collaboration on adopting it.
But if you do like our work, I hope that you stick with us as we create the best Nix experience we possibly can.
I want to deliver that in the most determined and focused way we can.
In sum, here's is what you need to know:
**Determinate Nix Installer will only install Determinate Nix no sooner than January 1, 2026**.
To that end, we'll be shipping these changes over the next few weeks:
1. We'll update our [documentation][installer-docs] to only show instructions for Determinate Nix.
1. Starting with the next release, Determinate Nix Installer and the [`DeterminateSystems/nix-installer-action`][installer-action] on GitHub will start warning users of this change with a link to this intent-to-ship blog post.
1. The installer will accept a new flag, `--prefer-upstream-nix`, that tells the installer that you specifically want upstream Nix for automated installations.
Then, starting on **November 10, 2025**:
1. Running the installer like this with the command below will switch to installing Determinate Nix unless you apply the `--prefer-upstream-nix` flag:
```shell title="This will install Determinate Nix beginning November 10, 2025"
curl -fsSL https://install.determinate.systems/nix | \
sh -s -- install --no-confirm
```
1. Our installer Action, [`DeterminateSystems/nix-installer-action`][installer-action], will switch to installing Determinate Nix unless `determinate: false` is passed.
Finally, starting no sooner than **January 1, 2026**:
1. Determinate Nix Installer will no longer prompt for Determinate Nix and will always install Determinate Nix.
The `--prefer-upstream-nix` flag will no longer have an effect.
1. [`DeterminateSystems/nix-installer-action`][installer-action] will also always install Determinate Nix.
I don't relish this choice but I think it's the right way to go.
Why?
Because we as a company neither own nor control upstream Nix, and that's a good thing.
But we want our installer to deliver something that we do control and are directly responsible for.
The Action will continue to warn users about this intent-to-ship change, which is declared in [this GitHub issue][issue].
Please contact us at [support@determinate.systems][support] if you have feedback about our intentions or our plan of action.
[better]: /blog/we-want-to-make-nix-better
[det-nix]: https://docs.determinate.systems/determinate-nix
[installer]: https://github.com/DeterminateSystems/nix-installer
[installer-action]: https://github.com/DeterminateSystems/nix-installer-action
[installer-docs]: https://github.com/determinateSystems/nix-installer?tab=readme-ov-file#determinate-nix-installer
[installer-fork]: https://github.com/NixOS/experimental-nix-installer
[installer-wg]: https://discourse.nixos.org/t/nix-installer-workgroup/21495
[issue]: https://github.com/DeterminateSystems/nix-src/issues/201
[lazy-trees]: /blog/changelog-determinate-nix-367
[linux-builder]: /blog/changelog-determinate-nix-384
[parallel-eval]: /blog/changelog-determinate-nix-3111
[stable]: /blog/determinate-nix-30
[support]: mailto:support@determinate.systems
[upstream]: https://github.com/NixOS/nix
---
**Parallel evaluation comes to Determinate Nix**
Published: September 5, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-3111
Determinate Nix 3.11.1 is out and it delivers [*parallel evaluation*](#parallel-evaluation), a long-awaited improvement, to our users.
In many real-world cases we've seen evaluation times **cut in half** or more.
But that's not all: we've drastically improved the [default `nix flake init` template](#default-template) and [fixed build cancellation on macOS](#build-cancellation).
Let's take a closer look.
### Parallel evaluation
Parallel evaluation (or *parallel eval* for short) is possibly the most exciting change we've shipped in months.
It promises to speed up a wide range of Nix operations by distributing work across multiple processors.
We plan to introduce parallel eval incrementally on a per-operation basis over the coming weeks and months.
We've started with these commands, which can now evaluate Nix expressions in parallel:
- `nix search`
- `nix flake check`
- `nix flake show`
- `nix eval --json`
As a real-world example, parallel eval has significantly reduced evaluation time for a flake we use internally that outputs multiple NixOS configurations.
How significant?
For this set of NixOS configurations, evaluation time was cut in half, from over 20 seconds to under 9 seconds.
Some other examples:
* `nix search "https://flakehub.com/f/NixOS/nixpkgs/0.1" cowsay --no-eval-cache` used to take 15 seconds and now takes 6 seconds (a speedup of 2.5x)
* `nix flake check github:NixOS/hydra/7de71224795804b3f5e6aa5a038d8143ba6fc415 --no-build --no-eval-cache` used to take 20 seconds and now takes less than 5.5 seconds (a speedup of 3.7x)
Parallel eval is currently in **developer preview**, and we've turned it on for our [Trailblazers][discord].
If you'd like to try it right away, set `eval-cores` to 0 in your [`/etc/nix/nix.custom.conf`][custom-config]:
```ini title="/etc/nix/nix.custom.conf"
eval-cores = 0 # Evaluate across all cores
```
### `builtins.parallel` \{#builtins-parallel}
With this release, we've also introduced a new builtin, `builtins.parallel`, that enables users to explicitly parallelize evaluation within a Nix expression.
Using `builtins.parallel` can drastically improve the performance of projects using [import-from-derivation][ifd] (IFD).
It reduced the evaluation time of `github:edolstra/parallel-eval-test#ifd`, for example, from 50 seconds to 10 seconds.
Using this new builtin requires turning on an additional experimental feature:
```ini title="/etc/nix/nix.custom.conf"
extra-experimental-features = parallel-eval
```
`builtins.parallel` could have its semantics changed significantly or even be removed from Determinate Nix during the developer preview.
### A useful `nix flake init` template default \{#default-template}
Nix's default flake template is [extremely bare bones](https://github.com/NixOS/templates/blob/ad0e221dda33c4b564fad976281130ce34a20cb9/trivial/flake.nix), and not a useful starting point.
Determinate Nix now uses [a more fleshed out default template](https://github.com/DeterminateSystems/flake-templates/blob/8af99b99627da41f16897f60eb226db30c775e76/default/flake.nix), including targeting multiple systems.
### Build cancellation is repaired on macOS \{#build-cancellation}
A recent macOS update changed how signals are handled by Nix and broke using Ctrl-C to stop a build.
Determinate Nix on macOS correctly handles these signals and stops the build.
## How to get Determinate Nix \{#install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.11.1 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.11.1"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v5
- uses: DeterminateSystems/flake-checker-action@main
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[custom-config]: https://docs.determinate.systems/determinate-nix/#determinate-nix-configuration
[det-nix]: https://docs.determinate.systems/determinate-nix
[discord]: https://determinate.systems/discord
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[ifd]: https://nix.dev/manual/nix/latest/language/import-from-derivation
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.11.1/aarch64-linux
[nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.11.1/x86_64-linux
---
**Changelog: build-time flake inputs and unauthenticated upgrades**
Published: August 27, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-390
Determinate Nix 3.9.0 brings an important optimization to flakes: [*build-time flake inputs*](#build-time-inputs).
These [inputs][flake-inputs] are sources that have no Nix expressions of their own and aren't required at evaluation time, such as some non-Nix Git repos.
When you mark a flake input as a build-time input, Nix downloads the source only when it's needed, at build time, and never before, which in turn provides generally much speedier and cleaner evaluation.
This is an exciting improvement because it provides the most benefit to precisely those folks who use flakes most intensively.
The other major change in 3.9.0 is that upgrading Determinate [no longer](#logged-out-upgrades) requires you to be logged into [FlakeHub].
A nice quality-of-life boost to accompany build-time inputs.
As a reminder: the [native Linux builder][linux-builder] in Determinate Nix enables macOS users to build for both ARM *and* x86 Linux with *zero* configuration.
The native Linux builders are currently in **developer preview** mode and will slowly be rolled out to Determinate Nix users over the coming weeks.
But if you're eager to try it out now, reach out to us at [support@determinate.systems][support] and include your [FlakeHub] username.
As always, don't hesitate to reach out to us with questions or feedback on [Discord] or via email at [hello@determinate-systems][email].
## Build-time flake inputs \{#build-time-inputs}
Some of our users have flakes with hundreds or even thousands of flake inputs.
In those cases, it can be painfully slow for Nix to fetch all the inputs during evaluation of the flake.
Determinate Nix now offers an experimental feature that, when enabled, makes Nix defer fetching those inputs until dependent derivations are actually built.
This feature is currently in **developer preview**.
If you'd like to try it, add this experimental feature to your custom Determinate Nix configuration at [`/etc/nix/nix.custom.conf`][custom-config]:
```ini title="/etc/nix/nix.custom.conf"
extra-experimental-features = build-time-fetch-tree
```
Then, change one of your inputs to be fetched at build time:
```nix title="flake.nix" {6-7}
{
inputs.nonNixInput = {
type = "github";
owner = "my-org";
repo = "non-nix-repo";
flake = false; # currently required
buildTime = true;
};
}
```
It's important to note that build-time fetching is performed by the Nix daemon and not by the Nix client.
This means that authenticated fetches only succeed if the daemon has access to any required credentials, such as GitHub access tokens.
Fetching Git repositories over SSH is currently not supported.
We may change the semantics of this feature at any time during its developer preview.
We don't plan to open pull request to the [upstream Nix repo][upstream-nix] until we're confident in its semantics and implementation.
That means that flakes that take advantage of this feature won't yet be compatible with upstream Nix and users should carefully consider flake compatibility before publishing any flakes that use this feature.
Let us know what you think on [Discord] or via email at [hello@determinate-systems][email].
## Corrected inconsistent behavior of `nix flake check` \{#nix-flake-check}
Users reported that the `nix flake check` command wouldn't consistently validate the entire flake.
We've fixed this issue and improved our testing around `nix flake check`.
## Say goodbye to slow Determinate Nix Installer downloads \{#installer-retries}
The Determinate Nix Installer now cancels and restarts downloads if the connection appears stalled.
Specifically, the connection is restarted if the connection is unable to transfer at least 250kb/sec over a 15-second period.
This is designed to avoid stalled and stuck connections and resume with a healthier backend.
Previously, users would occasionally see a stalled download take ten minutes or more.
We picked this threshold as it's approximately 20% of a typical and modern DSL connection.
If your network connection is normally below this rate, please get in touch and we'll tune it.
## Upgrade without logging in \{#logged-out-upgrades}
Previously, running [`determinate-nixd upgrade`][upgrade] required you to be logged into [FlakeHub].
With version 3.9.0, this is no longer necessary, which should streamline keeping up to date with the latest and greatest [Determinate Nix][det-nix].
## How to get Determinate Nix \{#install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.9.0 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.9.0"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- uses: DeterminateSystems/flake-checker-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[custom-config]: https://docs.determinate.systems/determinate-nix#determinate-nix-configuration
[det-nix]: https://docs.determinate.systems/determinate-nix
[discord]: https://determinate.systems/discord
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[email]: mailto:hello@determinate.systems
[flake-inputs]: https://zero-to-nix.com/concepts/flakes#inputs
[flakehub]: https://flakehub.com
[linux-builder]: /blog/changelog-determinate-nix-384
[nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.9.0/aarch64-linux
[nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.9.0/x86_64-linux
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[support]: mailto:support@determinate.systems
[upgrade]: https://docs.determinate.systems/determinate-nix#determinate-nixd-upgrade
[upstream-nix]: https://github.com/NixOS/nix
---
**Changelog: Determinate now has a nix-darwin module, plus loads of performance perks**
Published: August 22, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-386
I forgot to publish the release notes for 3.8.5.
Oops 😬.
So now you get a two-for-one deal instead!
We've recently shipped a *ton* of performance improvements, some small, some pretty significant.
This release also improved the [native Linux builders][linux-builder] in some scenarios, especially around network access.
As a reminder: the native Linux builders in Determinate Nix enable macOS users to build for both ARM *and* x86 Linux with *zero* configuration.
The native Linux builders are currently in **developer preview** mode and will slowly be rolled out to Determinate Nix users over the coming weeks.
But if you're eager to try it out now, reach out to us at [support@determinate.systems][support] and include your [FlakeHub] username.
As always, don't hesitate to reach out to us with questions or feedback on [Discord] or via email at [hello@determinate-systems][email].
And now for the updates.
Let's start with a user-facing change folks have been really clamoring for.
## Determinate now has a nix-darwin module! \{#nix-darwin}
You can finally configure Determinate Nix with [nix-darwin].
Add the nix-darwin module and customize Determinate Nix to taste:
```nix title="flake.nix" {10-15}
{
inputs.determinate.url =
"https://flakehub.com/f/DeterminateSystems/determinate/3.8.6";
inputs.nix-darwin.url =
"https://flakehub.com/f/nix-darwin/nix-darwin/0";
outputs.darwinConfigurations.mac = inputs.nix-darwin.lib.darwinSystem {
inherit system;
modules = [
inputs.determinate.darwinModules.default
{
determinate-nix.customSettings = {
flake-registry = "/etc/nix/flake-registry.json";
};
}
];
};
}
```
Notice the `customSettings` parameter, which enables you to pass in whatever [custom Nix configuration][custom-config] you like.
That configuration is then written to `/etc/nix/nix.custom.conf` for you.
## Interactive UX
### Nix's evaluation caching no longer blocks other evaluators \{#eval-cache}
Nix's evaluation cache used to block other Nix processes from evaluating.
This was most often noticed with tools like [direnv].
Determinate Nix doesn't do that, instead enabling you to evaluate many projects in parallel.
### More responsive tab completion \{#faster-tab-completion}
Tab completion now implies the `--offline` flag, which disables most network requests.
Previously, tab-completing Nix arguments would attempt to fetch sources and access binary caches.
Operating in offline mode improves the interactive experience of Determinate Nix when tab completing.
### ZFS users: we fixed the mysterious stall \{#no-zfs-stall}
Opening the Nix database is usually instantaneous but would occasionally impose several seconds of mysterious latency.
This was due to a quirk in [ZFS] that was *quite* recently fixed in that project.
Determinate Nix works around the issue as well, eliminating the frustrating random stall when running `nix` commands.
## Faster evaluation with more efficient I/O \{#faster-evaluation}
### "Unpacking into the Git cache" is much faster \{#unpacking-git-cache}
Unpacking sources into the user's cache with Determinate Nix now takes 1/2 to 1/4 of the time it used to.
Previously, Nix serially unpacked sources into the cache.
This change takes better advantage of our users' hardware by parallelizing the import.
Real-life testing shows that an initial Nixpkgs import takes 3.6 seconds on Linux when it used to take 11.7 seconds, which is **8.1 seconds less**.
## Parallelized copying of sources to the daemon \{#parallelized-copying}
Determinate Nix's evaluator no longer blocks evaluation when copying paths to the store.
Previously, Nix would pause evaluation when it needed to add files to the store.
Now, the copying is performed in the background allowing evaluation to proceed.
## Faster Nix evaluation
### Eliminated *thousands* of duplicate Nix daemon queries \{#daemon-queries}
Determinate Nix more effectively caches store path validity data within a single evaluation.
Previously, the Nix client would perform many thousands of extra Nix daemon requests.
Each extra request takes real time, and this change reduced a sample evaluation by over 12,000 requests.
### Parallelized memory garbage collection \{#parallel-gc}
We changed the "marking" process of the evaluator's garbage collector to use multiple threads.
Determinate Nix more quickly collects garbage across the large heap that Nixpkgs and NixOS tends to create.
This change has cut over five seconds from `nix search` on nixpkgs, from 24.3 seconds to 18.9 seconds.
### Fewer daemon queries, fewer daemon reconnects \{#fewer-daemon-calls}
We fixed a bug with [lazy trees][lazy-trees] that caused the Nix client to frequently reconnect to the daemon.
This issue caused a fairly significant regression in the performance of lazy trees in some cases.
As part of this change, we also reduced the number of queries the client needs to make in the first place.
## Other changes
* We fixed an issue where the Nix evaluation cache was a bit too eager and failed with `don't know how to recreate store derivation`.
* Determinate Nix is now fully formatted by [clang-format], making it easier than ever to contribute to the project.
* Determinate Nix now uses `main` as our development branch, moving away from `detsys-main`.
* Determinate Nix is now based on upstream Nix 2.30.2.
## How to get Determinate Nix \{#install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.8.6 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.8.6"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- uses: DeterminateSystems/flake-checker-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[clang-format]: https://clang.llvm.org/docs/ClangFormat.html
[custom-config]: https://docs.determinate.systems/determinate-nix#determinate-nix-configuration
[det-nix]: https://docs.determinate.systems/determinate-nix
[direnv]: https://github.com/direnv/direnv
[discord]: https://determinate.systems/discord
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[email]: mailto:hello@determinate.systems
[flakehub]: https://flakehub.com
[lazy-trees]: /blog/changelog-determinate-nix-367
[linux-builder]: /blog/changelog-determinate-nix-384
[nix-darwin]: https://github.com/nix-darwin/nix-darwin
[nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.8.6/aarch64-linux
[nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.8.6/x86_64-linux
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[support]: mailto:support@determinate.systems
[zfs]: https://wiki.archlinux.org/title/ZFS
---
**Changelog: a native Linux builder for macOS**
Published: August 5, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-384
[Determinate Nix][det-nix] version [3.8.4][det-nix-version] is now available and we're extremely happy to report that it introduces one of our most vital features yet: a **native Linux builder for macOS**.
That's right: with Determinate Nix you can now build Linux derivations on macOS without pulling from a cache or running a remote builder, a [Docker] image, or a Linux virtual machine—a massive step up in cross-platform collaboration and a testament to Determinate Systems' deep commitment to making [macOS][macos-tag] a first-class platform for Nix.
Imagine being on your MacBook Air and being able to build an ARM Linux package without even thinking about it:
```shell title="Build an ARM Linux package on macOS"
nix build nixpkgs#legacyPackages.aarch64-linux.cowsay
```
Or even an x86 Linux package:
```shell title="Build an x86 Linux package on macOS"
nix build nixpkgs#legacyPackages.x86_64-linux.cowsay
```
This is now possible with Determinate Nix.
The native Linux builder is currently in **developer preview** mode and will slowly be rolled out to Determinate Nix users over the coming weeks.
But if you're eager to try it out now, reach out to us at [support@determinate.systems][support] and include your [FlakeHub] username.
## Why we did it \{#why}
Why a native Linux builder for Nix?
Because fundamentally, all Nix [derivations] are [system specific][system-specificity], which means that, generally speaking, Nix can only build for the current host system (the one you get when you evaluate [`builtins.currentSystem`][current-system]).
Are you on a recent macOS system?
Then you can only build for `aarch64-darwin`.
Want to build something for Linux, like a [NixOS][nixos-nixpkgs] system or a [Docker] image?
Then you generally need to build on a [remote builder][remote-builds], pull from a [cache][caching], or even spin up a [Docker] container.
These solutions do work but they all require varying levels of setup and they're far less ergonomic than a straightforward `nix build`.
## How it works
In a nutshell, Determinate Nix uses macOS's recently introduced built-in [Virtualization framework][virtualization].
[Determinate Nixd][dnixd], a helper tool for Determinate Nix, calls some [Swift] code that uses that framework to [realise][realisation] your derivation.
In terms of configuration, the Determinate Nix CLI now has an `external-builders` experimental feature that needs to be enabled as well as an [`external-builders`][external-builders] setting that enables you to pass an external builders configuration as JSON (note that "external" in this case means external to Nix and *not* remote).
Here's an example Nix configuration that would delegate Linux builds to the Virtualization framework via Determinate Nixd:
```ini title="/etc/nix/nix.conf"
extra-experimental-features = external-builders
external-builders = [{"systems":["aarch64-linux","x86_64-linux"],"program":"/usr/local/bin/determinate-nixd","args":["builder"]}]
```
Please note that this configuration snippet does *not* yet work for most Determinate Nix users, as we'll be rolling this feature out incrementally over time (and only to macOS systems, of course).
At any time, you can see if this feature is enabled by either checking your configuration at `/etc/nix/nix.conf` for lines like the ones above or, better, by running this Determinate Nixd command:
```shell title="Ask Determinate Nixd if the Linux builder is available"
determinate-nixd version
```
If you see the message `The feature native-linux-builder is enabled` then you can start building for Linux now.
You can see some of the work that went into the Nix side of this in these pull requests:
As always, don't hesitate to reach out to us with questions or feedback on [Discord] or via email at [hello@determinate-systems][email].
Finally, special thanks go to Determinate Systems engineer [Cole Helbling][sep], whose concerted efforts were the driving force behind this feature.
## How to get Determinate Nix \{#install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.8.4 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.8.4"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- uses: DeterminateSystems/flake-checker-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[caching]: https://zero-to-nix.com/concepts/caching
[current-system]: https://nix.dev/manual/nix/latest/language/builtins.html#builtins-currentSystem
[derivations]: https://zero-to-nix.com/concepts/derivations
[det-nix]: https://docs.determinate.systems/determinate-nix
[det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.8.4
[discord]: https://determinate.systems/discord
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[docker]: https://docker.com
[email]: mailto:hello@determinate.systems
[external-builders]: https://manual.determinate.systems/development/experimental-features.html?highlight=external-builders#external-builders
[flakehub]: https://flakehub.com
[macos-tag]: /blog/tags/macos
[nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.8.4/aarch64-linux
[nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.8.4/x86_64-linux
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[nixos-nixpkgs]: https://github.com/NixOS/nixpkgs/tree/master/nixos
[realisation]: https://zero-to-nix.com/concepts/realisation
[remote-builds]: https://nix.dev/manual/nix/latest/advanced-topics/distributed-builds
[sep]: /people/cole-helbling
[support]: mailto:support@determinate.systems
[swift]: https://swift.org
[system-specificity]: https://zero-to-nix.com/concepts/system-specificity
[virtualization]: https://developer.apple.com/documentation/virtualization
---
**Changelog: Determinate Nix 3.8.1 with important security updates**
Published: July 12, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-381
[Determinate Nix][det-nix] version [**3.8.1**][det-nix-version] is now available, including important security improvements.
This release follows the revocation of [Determinate Nix 3.8.0][det-nix-3-8-0] after discovering a security regression for macOS users.
All users on macOS should upgrade to Determinate Nix 3.8.1 as soon as possible to address the issue.
We will publish further details in a follow-up post.
Note that platform and security notices are also published to our [status page][status-page], and users are encouraged to subscribe for updates.
For further questions, feel free to contact [support].
## How to get Determinate Nix \{#install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.8.1 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.8.1"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- uses: DeterminateSystems/flake-checker-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[det-nix-3-8-0]: /blog/changelog-determinate-nix-380/
[det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.8.1
[det-nix]: https://docs.determinate.systems/determinate-nix
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.8.1/aarch64-linux
[nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.8.1/x86_64-linux
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[status-page]: https://status.determinate.systems/
[support]: mailto:support@determinate.systems
---
**Changelog: a faster `nix flake check`, improved flake locks, and lazy trees rolled out to 20% of users**
Published: July 11, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-380
[Determinate Nix][det-nix] version [**3.8.0**][det-nix-version], based on version [2.30.0][nix-version] of upstream Nix, includes a variety of important improvements.
## An improved `nix flake check` \{#nix-flake-check}
The [`nix flake check`][nix-flake-check] command no longer downloads flake outputs in cases when no building is necessary.
This command validates that all of a flake's output successfully evaluate and that all outputs can thus be built.
If the outputs are available in a binary cache, then both properties have already been confirmed to be true, and `nix flake check` shouldn't need to re-evaluate.
Notably, downloading the output from the binary cache is not strictly necessary for the validation.
That's because `nix flake check` never created any garbage collection roots, and therefore makes no guarantees about the build product being in the local store.
Previously, `nix flake check` would download a flake output if the full build is available in a binary cache.
Some users may find that this change significantly reduces costly bandwidth and CI workflow time.
In our internal testing, we saw a reduction of several minutes in some of our repositories.
## Improved flake locking for transitive dependencies \{#transitive-dependencies}
Determinate Nix now re-locks all transitive dependencies when changing a [flake input][inputs]'s source URL.
This fixes an issue where Nix wouldn't re-lock those inputs in some scenarios and thus incorrectly used the old inputs' dependencies.
## A smarter `determinate-nixd upgrade` \{#determinate-nixd-upgrade}
If you try to upgrade Determinate Nix when you're already on the latest version, Determinate Nixd now exits instead of upgrading again.
This is a small improvement but a nice quality-of-life boost.
Some users also reported `determinate-nixd version` timing out the first time it's run after a reboot.
That should be fixed now as well.
## Lazy trees is rolled out to 20% of users \{#lazy-trees-rollout}
In the last release, we rolled out [lazy trees][lazy-trees] to 5% of Determinate Nix users.
After a successful 5% rollout, we're now expanding this group to 20% of users.
You can see if you're enrolled in the lazy trees rollout using the [`determinate-nixd version`][dnixd-version] command:
```shell title="Check if Determinate Nix is due for an upgrade"
determinate-nixd version
```
If Determinate Nix is up to date, you should see this:
```shell
Determinate Nixd daemon version: 3.8.0
Determinate Nixd client version: 3.8.0
You are running the latest version of Determinate Nix.
The feature lazy-trees is enabled.
```
If you see any bugs or would like to opt out, please contact [support@determinate.systems][support].
Lazy trees is a massive usability and performance improvement to flakes and we're excited to see it roll out to thousands more users daily.
## How to get Determinate Nix \{#install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.8.0 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.8.0"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- uses: DeterminateSystems/flake-checker-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[det-nix]: https://docs.determinate.systems/determinate-nix
[det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.8.0
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[dnixd-version]: https://docs.determinate.systems/determinate-nix#determinate-nixd-version
[inputs]: https://zero-to-nix.com/concepts/flakes#inputs
[lazy-trees]: https://dtr.mn/lazy-trees
[nix-flake-check]: https://manual.determinate.systems/command-ref/new-cli/nix3-flake-check.html
[nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.8.0/aarch64-linux
[nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.8.0/x86_64-linux
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[support]: mailto:support@determinate.systems
[nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.30
---
**Changelog: faster CI, deep flake overrides, and a better `nix store delete` in Determinate Nix 3.7.0**
Published: July 4, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-370
Celebrate the Fourth of July with the release of [Determinate Nix][det-nix] version [**3.7.0**][det-nix-version]!
We've shipped several Nix performance and UX improvements and we're also excited to roll out [lazy trees][lazy-trees] to more users.
## Prefetch flake inputs in parallel
This release introduces the `nix flake prefetch-inputs` command to Determinate Nix.
[Flake inputs][inputs] are typically fetched "just in time."
That means that Nix fetches flake inputs as needed during the evaluation process, which can slow down evaluation in projects with many flake inputs.
Ideally, though, Nix could avoid such slow-downs by fetching flake inputs **prior to evaluation**.
The new `nix flake prefetch-inputs` command does precisely that, fetching all flake inputs in parallel so that they're already available when you need to evaluate any expression.
We expect that running this new command before building things with Nix should dramatically improve evaluation performance for most projects, especially in CI.
Note that projects with many unused flake inputs may not benefit from this change since the new command fetches every inputs whether they're used or not.
We plan to run some experiments in GitHub Actions to see if and how much this helps.
## Deep flake input overrides now work as expected
An override like this...
```nix
{
inputs.foo.inputs.bar.inputs.nixpkgs.follows = "nixpkgs";
}
```
...used to implicitly set `inputs.foo.inputs.bar` to `flake:bar`, which led to an unexpected error like this:
```shell
error: cannot find flake 'flake:bar' in the flake registries
```
With this release, Nix no longer creates a parent override (like for `foo.bar` in the example above) if it doesn't set an explicit `ref` or `follows` attribute; instead, Nix only recursively applies its child overrides.
This closes four upstream Nix issues.
Check out [DeterminateSystems/nix-src#95][nix-src-95] for more details and a snazzy diagram!
## `nix store delete` now shows you why deletion was not possible
Here's some example error output:
```shell
error: Cannot delete path '/nix/store/6fcrjgfjip2ww3sx51rrmmghfsf60jvi-patchelf-0.14.3'
because it's referenced by the GC root '/home/eelco/Dev/nix-master/build/result'.
error: Cannot delete path '/nix/store/rn0qyn3kmky26xgpr2n10vr787g57lff-cowsay-3.8.4'
because it's referenced by the GC root '/proc/3600568/environ'.
error: Cannot delete path '/nix/store/klyng5rpdkwi5kbxkncy4gjwb490dlhb-foo.drv'
because it's in use by '{nix-process:3605324}'.
```
## Lazy-tree improvements
Improved lazy tree evaluation caching for flakes accessed with a `path` flakeref.
## Lazy trees on its way to general availability
Since the last release, we've finished preparation for rolling out lazy tree to more users.
We're going to start enabling lazy trees for about 5% of Determinate Nix users on version 3.7.0 outside of CI.
You can see if you're enrolled in the lazy trees rollout using the [`determinate-nixd version`][dnixd-version] command:
```shell title="Check if Determinate Nix is due for an upgrade"
determinate-nixd version
```
If Determinate Nix is up to date, you should see this:
```shell
Determinate Nixd daemon version: 3.7.0
Determinate Nixd client version: 3.7.0
You are running the latest version of Determinate Nix.
The feature lazy-trees is enabled.
```
If you'd like to opt out, please contact [support@determinate.systems][support].
We're excited for more folks to experience the, frankly, huge performance benefits of lazy trees in their workflows.
## How to get Determinate Nix \{#install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.7.0 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.7.0"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our NixOS ISO ([NixOS installer for x86_64][nixos-iso-x86], [NixOS installer for ARM][nixos-iso-arm]) with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- uses: DeterminateSystems/flake-checker-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.7.0
[det-nix]: https://docs.determinate.systems/determinate-nix
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[dnixd-version]: https://docs.determinate.systems/determinate-nix#determinate-nixd-version
[inputs]: https://zero-to-nix.com/concepts/flakes#inputs
[lazy-trees]: https://dtr.mn/lazy-trees
[nixos-iso-arm]: https://install.determinate.systems/nixos-iso/tag/v3.7.0/aarch64-linux
[nixos-iso-x86]: https://install.determinate.systems/nixos-iso/tag/v3.7.0/x86_64-linux
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[support]: mailto:support@determinate.systems
[nix-src-95]: https://github.com/DeterminateSystems/nix-src/issues/95
---
**Changelog: lazy trees and security improvements**
Published: June 24, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-367
Determinate Nix [**v3.6.7**][det-nix-version] includes important security patches and users should [upgrade](#install) as soon as possible.
## Security contents
This release fixes GHSA-g948-229j-48j3, a vulnerability reported by [Snyk]'s Security Labs team.
The vulnerability allowed a user who could build software with the Nix daemon and coordinate running a program on the host to elevate to root.
The upstream Nix project has not fully published the security advisory but we will update this post when they do.
## Lazy trees now saves NAR hashes to the `flake.lock`
[Lazy trees][lazy-trees] now produces `flake.lock` files with NAR hashes.
Here's an example command that exhibits this behavior:
```shell title="Generate a lockfile with NAR hashes"
nix flake update --commit-lock-file
```
The previous behavior omitted NAR hashes from the `flake.lock`, which meant that adopting lazy trees required full buy-in from all users.
Now, you can more confidently enable lazy trees in CI and other workflows without first enabling lazy trees for all users.
You can restore the previous behavior by setting `lazy-locks` to `true`.
## Improved caching with impure evaluation and lazy trees
Users have reported less effective caching when using lazy trees and impure evaluation.
This release improves the caching side of things and should improve things substantially.
## Lazy trees on its way to general availability
We have laid the ground-work to move lazy trees from feature preview into general availability.
For the next phase in this process we will start progressively enabling lazy trees for users.
If you'd like to opt out, please contact [support@determinate.systems][support].
You can see if you're enrolled in the lazy trees using the [`determinate-nixd version`][dnixd-version] command:
```shell title="Check if Determinate Nix is due for an upgrade"
determinate-nixd version
```
If Determinate Nix is up to date, you should see this:
```shell
Determinate Nixd daemon version: 3.6.7
Determinate Nixd client version: 3.6.7
You are running the latest version of Determinate Nix.
The feature lazy-trees is enabled.
```
## How to get Determinate Nix \{#install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.6.7 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.6.7"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our [NixOS ISO][nixos-iso] with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- uses: DeterminateSystems/flake-checker-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.6.7
[det-nix]: https://docs.determinate.systems/determinate-nix
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[dnixd-version]: https://docs.determinate.systems/determinate-nix#determinate-nixd-version
[lazy-trees]: https://dtr.mn/lazy-trees
[nixos-iso]: https://github.com/DeterminateSystems/nixos-iso/releases/tag/v3.6.7
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[snyk]: https://labs.snyk.io/security-labs
[support]: mailto:support@determinate.systems
---
**FlakeHub now supports Semaphore CI**
Published: June 19, 2025
URL: https://determinate.systems/blog/semaphore-ci
Providing synchronized [development environments][envs] across continuous integration/delivery (CI/CD) platforms and developer workstations has always been one of Nix's most compelling use cases.
And so we're excited to announce that [FlakeHub] now supports [Semaphore CI][semaphore] as our third major CI platform after [GitHub Actions][actions] and [GitLab CI][gitlab].
Semaphore is a powerful open source CI/CD platform that you can [run on your own infrastructure][semaphore-self-hosting] or [in the cloud][semaphore-cloud].
It features a [lovely UI][semaphore-canvas] for visualizing workflows, a built-in [observability suite][semaphore-observability], and much more.
We encourage you to check it out.
With this new support, you can now [publish flakes][publish], including [private flakes][private-flakes], to FlakeHub and push store paths to [FlakeHub Cache][flakehub-cache] inside your Semaphore CI runs.
Here is an Semaphore workflow configuration that does both:
```yaml title=".semaphore/semaphore.yml" collapse={8-80}
version: v1.0
name: Push store paths to FlakeHub Cache and publish flake release on FlakeHub
agent:
machine:
type: f1-standard-4
os_image: ubuntu2404
blocks:
- name:
dependencies: []
task:
prologue:
commands:
# Get the latest version of the repository's source code from GitHub
- checkout
jobs:
- name: Publish flake and cache package
commands:
# The flake's repository
- export FLAKEHUB_PUSH_REPOSITORY="$(echo "${SEMAPHORE_ORGANIZATION_URL}" | cut -d "." -f1 | cut -d '/' -f3)/${SEMAPHORE_PROJECT_NAME}"
# Environment variables for Magic Nix Cache, which automatically pushes Nix artifacts to FlakeHub Cache
- export MAGIC_NIX_CACHE_CLOSURE_URL="https://install.determinate.systems/magic-nix-cache-closure/branch/main/X64-Linux?ci=semaphore"
- export MNC_LISTEN="127.0.0.1:37515"
# Install Determinate Nix and start the Nix daemon
- curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux --determinate --no-confirm --init systemd
- . /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh
# Log in using the `determinate-nixd login` command (used by magic-nix-cache, substitutions)
- echo "${SEMAPHORE_OIDC_TOKEN}" | determinate-nixd login token --token-file /dev/stdin
# Acquire the `flakehub-push` executable
- curl -L "${FLAKEHUB_PUSH_BINARY_URL}" | sudo tee /usr/bin/flakehub-push &>/dev/null
- sudo chmod +x /usr/bin/flakehub-push
# Acquire the `magic-nix-cache` executable
- export MNC_CLSR="$(curl -L "${MAGIC_NIX_CACHE_CLOSURE_URL}" | xz -d | sudo "$(which nix-store)" --import | tail -n1 | head -n1)"
- sudo ln -sf "${MNC_CLSR}/bin/magic-nix-cache" /usr/bin/magic-nix-cache
- magic-nix-cache --help
# Stage login credentials for `flakehub-push`
- export FLAKEHUB_PUSH_OIDC_TOKEN="${SEMAPHORE_OIDC_TOKEN}"
# Start Magic Nix Cache
- export MNC_STARTUP_FILE="/tmp/mnc-startup"
- nohup magic-nix-cache --listen "${MNC_LISTEN}" --startup-notification-file "${MNC_STARTUP_FILE}" &>/tmp/mnc.log &
- |
(
STARTED=0
for n in {1..6}; do
if [ -e "${MNC_STARTUP_FILE}" ]; then
echo "magic-nix-cache daemon has successfully started up after ${n} attempt(s)"
STARTED=1
break
else
echo "waiting on magic-nix-cache daemon; on attempt ${n}"
sleep 2
fi
done
if [[ "${STARTED}" != "1" ]]; then
echo "The daemon did not start up within 60 seconds; exiting"
exit 1
fi
) || true
# Build a package output by the repository's Nix flake
- nix build ".#packages.x86_64-linux.default"
# Publish a flake release to FlakeHub if and only if it's a tag reference
- |
if [[ "${SEMAPHORE_GIT_REF_TYPE}" == "tag" ]]; then
flakehub-push \
--tag "$(cat "${SEMAPHORE_GIT_REF}" | cut -d '/' -f2)" \
--visibility private \
--include-output-paths
fi
# Stop Magic Nix Cache
- curl -XPOST "http://${MNC_LISTEN}/api/workflow-finish"
```
For more detailed information, check out the [Semaphore CI guide][semaphore-guide] in the [Determinate documentation][docs].
If you run into any trouble, get in touch with us [on Discord][discord] or drop us an email at [support@determinate.systems][support].
Enjoy!
[actions]: https://github.com/features/actions
[discord]: https://determinate.systems/discord
[docs]: https://docs.determinate.systems
[envs]: https://zero-to-nix.com/concepts/dev-envs
[flakehub]: https://flakehub.com
[flakehub-cache]: https://flakehub.com/cache
[gitlab]: https://docs.gitlab.com/ci
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[publish]: https://docs.determinate.systems/flakehub/publishing
[semaphore]: https://semaphore.io
[semaphore-canvas]: https://semaphore.io/product/canvas
[semaphore-cloud]: https://semaphore.io/cloud
[semaphore-guide]: https://docs.determinate.systems/guides/semaphore
[semaphore-observability]: https://semaphore.io/product/metrics-and-observability
[semaphore-self-hosting]: https://semaphore.io/product/self-hosted-agents
[support]: mailto:support@determinate.systems
---
**Changelog: macOS Tahoe, lazy trees, and docs in Determinate Nix 3.6.6**
Published: June 18, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-366
We're excited to announce the release of [Determinate Nix][det-nix] version [**3.6.6**][det-nix-version], based on version [2.29.0][nix-version] of upstream Nix.
## Update on macOS Tahoe
We've been testing our releases on macOS Tahoe internally since the first day of its release.
All of our data shows that Determinate Nix works great on the current Tahoe beta.
We'll continue to monitor the situation.
If you see something that we should take a look at, please leave a comment on [this GitHub issue](https://github.com/DeterminateSystems/determinate/issues/100).
## Update on lazy trees
We released [lazy trees][lazy-trees] in Determinate Nix 3.5.2, about a month ago, and since then hundreds of users have activated them.
Across the board, these users have reported much faster CI times and in some scenarios a speedup of 5x or more.
This 3.6.6 release includes additional performance and reliability improvements for lazy trees.
It's easy to turn them on by adding this single line to your `nix.custom.conf`:
```shell title="/etc/nix/nix.custom.conf"
lazy-trees = true
```
On NixOS, you can enable the NixOS option:
```nix title="/etc/nixos/configuration.nix"
{
nix.settings.lazy-trees = true;
}
```
For those keeping track, you may have noticed that we skipped 3.6.3 and 3.6.4.
We canceled the release of 3.6.3 and 3.6.4 after we identified some regressions around [FlakeHub authentication][flakehub-auth].
We've since improved our test suite to avoid this in the future.
## Even faster evaluation with lazy trees through improved input caching
Some users reported that flake inputs were not being cached as often as they expected.
[Eelco Dolstra][eelco] landed several pull requests to make this better, which has cut CI times for some users from over two minutes to under 20 seconds.
## Extended lazy trees test suite
We now run the full Nix test suite and the Nixpkgs library test suite with lazy trees enabled.
This revealed two bugs that are now fixed.
## `determinate-nixd status` now correctly checks the `netrc-file` setting on NixOS
Running a [`determinate-nixd status`][dnixd-status] check examines Nix's `netrc-file` configuration setting.
We added this check to help users who accidentally changed the setting, which breaks FlakeHub access.
On NixOS, we unfortunately used the wrong path for Nix and the check didn't work.
By the way: if you need to customize the netrc, use [`authentication.additionalNetrcSources`](https://docs.determinate.systems/determinate-nix#additionalnetrcsources) instead of changing the netrc file.
## Missing sandbox paths now have a sensible error message
Users with a misconfigured [`sandbox-paths`][sandbox-paths] configuration used to see a mysterious error about a missing file.
Users now get an error message that helps them solve the problem:
```shell
path '/etc/nix/some-missing-file' is configured as part of the `sandbox-paths` option, but is inaccessible
```
## The Determinate package on macOS can upgrade more systems
This used to fail on systems that used `nix profile` to manage the default profile.
We can now upgrade these systems as well.
## Assorted improvements
* When remote building with `--keep-failed`, we only show the "you can rerun" message if the derivation's platform is supported on the current machine: [DeterminateSystems/nix-src#87](https://github.com/DeterminateSystems/nix-src/pull/87)
* Lazy tree paths in messages are rendered without the `/nix/store/...` prefix in substituted source trees: [DeterminateSystems/nix-src#91](https://github.com/DeterminateSystems/nix-src/pull/91)
* We fixed built-in functions being registered twice: [DeterminateSystems/nix-src#97](https://github.com/DeterminateSystems/nix-src/pull/97)
* We fixed the link to `builders-use-substitutes` documentation for the `builders` option: [DeterminateSystems/nix-src#102](https://github.com/DeterminateSystems/nix-src/pull/102)
* We updated error messages to remove what we call the "fake future tense:" [DeterminateSystems/nix-src#92](https://github.com/DeterminateSystems/nix-src/pull/92)
* We added missing help text to some `determinate-nixd auth login` options.
## How to get Determinate Nix
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.6.6 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.6.6"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our [NixOS ISO][nixos-iso] with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- uses: DeterminateSystems/flake-checker-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.6.6
[det-nix]: https://docs.determinate.systems/determinate-nix
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[dnixd-status]: https://docs.determinate.systems/determinate-nix#determinate-nixd-status
[eelco]: /people/eelco-dolstra
[flakehub-auth]: https://docs.determinate.systems/flakehub/concepts/authentication
[lazy-trees]: /blog/changelog-determinate-nix-352
[nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.29
[nixos-iso]: https://github.com/DeterminateSystems/nixos-iso/releases/tag/v3.6.6
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[sandbox-paths]: https://nix.dev/manual/nix/latest/command-ref/conf-file.html#conf-sandbox-paths
---
**Bringing back the Magic Nix Cache Action**
Published: June 13, 2025
URL: https://determinate.systems/blog/bringing-back-magic-nix-cache-action
A while back, we [announced][announcement] the end of life for our [Magic Nix Cache Action][mnca], a [GitHub Action][actions] that uses Actions' [built-in cache][gha-cache] to share the results of Nix builds between workflow runs.
Nix users responded quite positively to getting a viable Nix binary cache for free and with zero configuration and we were happy to provide it to them.
And then we were sad when GitHub announced that the Actions cache was being [deprecated][github-blog-post] in favor of a new (and undocumented) API.
Today, we're happy to announce that **Magic Nix Cache Action is back**.
GitHub user [jchv] submitted a heroic [pull request][v2-pr] to make [Magic Nix Cache][mnc] communicate with the new version of Actions' cache API.
That means that as of the [most recent release][v2-release] of Magic Nix Cache you can once again use the Magic Nix Cache Action in your workflows.
Here's an example configuration:
```yaml title="Example Magic Nix Cache Action workflow" {10}
jobs:
nix-build:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/magic-nix-cache-action@main
- name: Build and cache with Nix
run: nix build
```
## Limitations
Please be aware, though, that Magic Nix Cache Action continues to have some deep limitations:
1. It provides caching only between runs of a specific workflow in a specific repository and thus doesn't serve as a standard Nix binary cache available across a wide variety of clients.
That means that you can't, for example, cache some Nix package builds in Actions and make those available to you and your teammates.
2. GitHub Actions' cache API is *not* a purpose-built Nix binary cache.
Unsurprisingly, its performance is decent but not great.
For most use cases, you should opt for [FlakeHub Cache][cache] instead, which offers true all-purpose binary caching and dramatically better performance and use Magic Nix Cache Action only when you want better performance between CI runs.
## An important caveat
On a similarly sober note, we do need to make something clear.
As I said above, the approach provided by [jchv] is based on a reverse engineering of GitHub's new caching API.
GitHub has *not* published the [Protobuf] sources for that API and that means both that there's a good bit of guesswork here and that the API could change at any time, which could in turn make the new approach nonviable.
We at Determinate Systems do hope that this will continue to work for years and years but the rug could in principle be pulled at any time.
C'est la vie.
We'd like to express our most sincere gratitude to [jchv] for their help.
Happy (free) caching, everyone!
[actions]: https://github.com/features/actions
[announcement]: /blog/magic-nix-cache-free-tier-eol
[cache]: https://flakehub.com/cache
[gha-cache]: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows
[github-blog-post]: https://github.blog/changelog/2024-09-16-notice-of-upcoming-deprecations-and-changes-in-github-actions-services
[jchv]: https://github.com/jchv
[mnc]: https://github.com/DeterminateSystems/magic-nix-cache
[mnca]: https://github.com/DeterminateSystems/magic-nix-cache-action
[protobuf]: https://protobuf.dev
[v2-pr]: https://github.com/DeterminateSystems/magic-nix-cache/pull/139
[v2-release]: https://github.com/DeterminateSystems/magic-nix-cache/releases/tag/v0.1.5
---
**Changelog: docs, diagnostics, and resilience improvements**
Published: June 5, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-362
[Determinate Nix][det-nix] version [**3.6.2**][det-nix-version], based on version [2.29.0][nix-version] of upstream Nix, includes a variety of small but substantial ergonomic improvements across a variety of domains.
## Migrate to or upgrade Determinate Nix *without* building it on NixOS
[NixOS users][nixos] can now migrate to and upgrade Determinate Nix by fetching it from a cache rather than needing to build it from source.
When initially installing Determinate Nix 3.6.2 using the [`determinate` flake][determinate-flake], make sure to pass a few extra flags to `nixos-rebuild`:
```shell title="Flags to pass when rebuilding on NixOS" {2-3}
sudo nixos-rebuild \
--option extra-substituters https://install.determinate.systems \
--option extra-trusted-public-keys cache.flakehub.com-3:hJuILl5sVK4iKm86JzgdXW12Y2Hwd5G07qKtHTOcDCM= \
--flake ... \
switch
```
This uses `install.determinate.systems`, our artifact server, as a binary cache for fetching Determinate Nix.
After the upgrade, users who are logged in to [FlakeHub] have [FlakeHub Cache][cache] in their substituters list.
If you are *not* logged in, you have `install.determinate.systems` in your substituters list.
This has been an annoying problem for quite some time because FlakeHub Cache requires authentication and offers no public, authentication-free caching.
We solved this problem by teaching `install.determinate.systems` how to be an authenticated proxy to FlakeHub Cache.
This public cache shows FlakeHub Cache's [access control rules][cache-auth] in action.
## Autocompletion for Determinate Nixd
You can enable auto-completion for [Determinate Nixd][dnixd] using the new `determinate-nixd completion` subcommand.
Pass your preferred shell—one of `bash`, `elvish`, `fish`, `powershell`, or `zsh`—and evaluate the result.
Here's an example for zsh:
```shell title="Load shell completion script for zsh"
eval "$(determinate-nixd completion zsh)"
```
Now when you run `determinate-nixd` and type in, say, "a" and hit **Tab**, it helpfully completes to `determinate-nixd auth`.
## Track down and fix "inefficient double copy" causes
When we recently released [lazy trees][lazy-trees] in feature preview, we added a warning for when a Nix expression forces Determinate Nix to copy the flake's source to the store unnecessarily.
Unfortunately, that warning was pretty useless because it didn't tell you *where* that happened.
Determinate Nix now identifies where in the source code the extra file copying happens.
So a command like `nix build .#extraCopy` would now produce a log message like this:
```shell title="Inefficient double copy message" {4}
warning: Copying '/Users/grahamc/src/github.com/DeterminateSystems/samples/' to the store again
You can make Nix evaluate faster and copy fewer files by replacing `./.` with the `self` flake input, or `builtins.path { path = ./.; name = "source"; }`
Location: /Users/grahamc/src/github.com/DeterminateSystems/samples/extra-copy.nix:12:11
error: Cannot build '/nix/store/2hh3855rfsabcj4gm2nl65a9la7xbcih-extra-copy.drv'.
```
## Eradicated symbols from `nix profile`
Historically, `nix profile` commands like `nix profile history` would print out symbols like `ε` and `∅` from set theory.
While these symbols *are* precisely defined, we don't want our users to *need* to brush up on mathematical logic to understand what is happening.
So `nix profile history` now prints a much more understandable summary.
Here's an example output for `nix profile history` with the current Determinate Nix:
```shell title="nix profile history output"
Version 1 (2024-12-06):
home-manager-path: (no version) added
Version 2 (2025-02-25) <- 1:
bar: (no version) added
foo: (no version) added
home-manager-path: (no version) removed
```
Now let's take a look at an older version of Nix.
This command...
```shell title="nix profile history for Nix version 2.18.0"
nix run github:NixOS/nix/2.18.0 -- profile history
```
...would produce output like this:
```shell title="nix profile history with set theory symbols"
Version 1 (2024-12-06):
home-manager-path: ∅ -> ε
Version 2 (2025-02-25) <- 1:
bar: ∅ -> ε
foo: ∅ -> ε
home-manager-path: ε -> ∅
```
We think the former is much more readily comprehensible to users.
## Clarify what `--keep-failed` plus remote building does
When you run `nix build --keep-failed`, the build directory won't be deleted from the system.
This is helpful for digging into the issue and solving the build problem.
Unfortunately, users are surprised to find that the directory doesn't exist if the build was run remotely.
The build directory was kept but it was kept *on the remote*.
We updated the build failure message to help the user out:
```shell title="Example log message"
copying path '/nix/store/cryiszlhhss2h40r3f9m60l92i88m2xd-builder-failing.sh' to 'ssh://localhost'...
note: keeping build directory '/tmp/nix-shell.8G497s/nix-build-failing.drv-16/build'
warning: `--keep-failed` will keep the failed build directory on the remote builder.
If the build's architecture matches your host, you can re-run the command with `--builders ''` to disable remote building for this invocation.
```
## Better diagnostics when trying to build a derivation on the wrong system type
Sometimes a Nix expression inadvertently depends on a [derivation] that needs to build on a different system.
When this happens, Determinate Nix now prints an error that includes the chain of derivations pulling in the mismatching build:
```shell {2-5}
% nix build .#badSystemNested
error: Cannot build '/nix/store/5vsaxi730yl2icngkyvn8wiflik5wfmq-bad-system.drv'.
Reason: required system or feature not available
Required system: 'bogus' with features {}
Current system: 'aarch64-darwin' with features {apple-virt, benchmark, big-parallel, nixos-test}
error: Cannot build '/nix/store/2gh9xc4bcq11vjxc80lw5hs3y4vnfvqy-nested-bad-system-bottom.drv'.
Reason: 1 dependency failed.
Output paths:
/nix/store/5kzq0mlvxhll9vayw22a5dy8j259vpp4-nested-bad-system-bottom
error: Cannot build '/nix/store/4lxwf00yrxnsraq00ga4dz4v5fxpk8pw-nested-bad-system-top.drv'.
Reason: 1 dependency failed.
Output paths:
/nix/store/glb9vi6zp3j0dlc3j1g1i46pkwbq7gx0-nested-bad-system-top
```
This error clearly indicates that the `bad-system` derivation was pulled in by `bad-system-top` via `bat-system-bottom`.
This is a dramatic improvement over the previous behavior, which excluded any context that would help identify the cause of the issue:
```
$ nix run github:NixOS/nix/2.18.0 -- build .#badSystemNested
error: a 'bogus' with features {} is required to build '/nix/store/5vsaxi730yl2icngkyvn8wiflik5wfmq-bad-system.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}
```
## Easily find where import-from-derivation is used
Import from derivation ("IFD") means using Nix to build a Nix expression that you then import.
A common example of this is [Haskell.nix](https://github.com/input-output-hk/haskell.nix).
IFD is a convenient "escape hatch" when packaging some language ecosystems, but comes at a serious cost of performance when using Nix.
Because of the severe performance impact, we strongly discourage IFD and even prohibit its use on packages published on FlakeHub.
One of our customers noted they are trying to cut back on how often they use IFD, but could not find *where* they used it.
We have introduced a new flag to Determinate Nix, `--trace-import-from-derivation` which makes it easier to find:
```shell
$ nix build --trace-import-from-derivation .#packages.aarch64-darwin.example
warning: built '/nix/store/8r2a9yk318q9nv8j3aqbd86r73j183fn-generated-nix-expression.drv^out' during evaluation due to an import from derivation
```
From here we can inspect the derivation to find what caused IFD to take place.
### Dramatically improved performance with `nix store copy-sigs`
We have increased the concurrency of `nix store copy-sigs`.
Copying signatures is not a CPU-heavy task but it was previously running one copy operation per core.
It now uses the `http-connections` setting to limit the concurrency, which is typically much higher, and defaults to 25.
We also updated the documentation to include a description and examples.
### Improved robustness while collecting garbage
The garbage collector would occasionally fail in the middle of cleaning up when encountering files that can not be deleted.
While Determinate Nix users are less likely to experience this problem due to managed garbage collection, it now identifies and logs the issue, and continues with garbage collection as normal.
### Configurable `max-jobs`
You can now change the `max-jobs` setting in `/etc/nix/nix.custom.conf`.
Previously, this value was forced to `auto`.
Now it defaults to `auto`, but you can update it to your own preference.
### Documented the `nix`-command replacement for `nix-store --query --deriver`
The documentation for `nix-store --query --deriver` now mentions this `nix`-command based alternative:
```shell
nix path-info --json ... | jq -r '.[].deriver'
```
## No more "unauthenticated" errors with FlakeHub Cache
The flake no longer unconditionally includes FlakeHub Cache in your list of active substituters.
That means you won't see 403 errors whenever you build something.
We fixed this issue a long time ago for macOS and Linux users, but forgot to delete it from the module for NixOS users.
Fixed now!
## `determinate.enable` for easy migrations
You can now add the [Determinate module][determinate-flake] to your global NixOS configuration and disable it on specific hosts.
This is useful while migrating to Determinate across a large fleet.
Thank you to GitHub user [getchoo] for [their PR][pr-97].
## `nix.settings` works again on NixOS
Determinate Nix manages its own `nix.conf` configuration, which can cause issues when trying to use `nix.settings` options on NixOS.
The Determinate module now configures NixOS to write out your extra settings to [`/etc/nix/nix.custom.conf`][custom-conf] instead.
Now you can use `nix.settings` and its structured configuration on NixOS without needing to adjust the module internals yourself.
## Determinate Nix Installer self-test
[Determinate Nix Installer][installer] performs a self-test at the end to measure how well it did.
Thanks to Josh Heinrichs from Shopify, the zsh self-test no longer takes over `/dev/tty`.
## How to get Determinate Nix
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.6.2 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.6.2"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos] or our [NixOS ISO][nixos-iso] with Determinate Nix pre-installed.
On GitHub Actions:
```yaml title=".github/workflows/nix-ci.yaml"
on:
pull_request:
workflow_dispatch:
push:
branches:
- main
jobs:
nix-ci:
runs-on: ubuntu-latest
# Include this block to log in to FlakeHub and access private flakes
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-cache-action@main
- uses: DeterminateSystems/flake-checker-action@main
- run: nix flake check
```
In Amazon Web Services:
```terraform title="aws.tf"
data "aws_ami" "detsys_nixos" {
most_recent = true
owners = ["535002876703"]
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
[cache]: https://docs.determinate.systems/flakehub/cache
[cache-auth]: https://docs.determinate.systems/flakehub/concepts/authentication
[custom-conf]: https://docs.determinate.systems/determinate-nix#determinate-nix-configuration
[derivation]: https://zero-to-nix.com/concepts/derivations
[det-nix]: https://docs.determinate.systems/determinate-nix
[det-nix-version]: https://github.com/DeterminateSystems/nix-src/releases/tag/v3.6.2
[determinate-flake]: https://github.com/DeterminateSystems/determinate
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[flakehub]: https://flakehub.com
[getchoo]: https://github.com/getchoo
[installer]: https://github.com/DeterminateSystems/nix-installer
[lazy-trees]: /blog/changelog-determinate-nix-352
[nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.29
[nixos]: https://docs.determinate.systems/guides/advanced-installation#nixos
[nixos-iso]: https://github.com/DeterminateSystems/nixos-iso/releases/tag/v3.6.2
[pr-97]: https://github.com/DeterminateSystems/nix-src/pull/97
---
**Changelog: improved support for self-hosted GitHub Actions runners**
Published: May 29, 2025
URL: https://determinate.systems/blog/changelog-improved-self-hosted-determinate-nix-action
The [`determinate-nix-action`][determinate-nix-action], which we recently [announced][announcement], now works on most self-hosted CI environments in GitHub Actions.
These once-problematic environments should now work just fine:
* [Kubernetes]
* [Actions Runner Controller][arc] (ARC)
* [act]
Previously, our Action used [Docker] as a process supervision daemon, but this proved to be overcomplicated and buggy in certain cases.
The problem is that any Docker-oriented approach only works if the Docker daemon's root filesystem is identical to the execution environment's root filesystem—an assumption that doesn't always hold in reality.
Our new method is much simpler.
We now spawn the [Determinate Nix daemon][dnixd] in the background and shut it down when the job shuts down.
This new approach works in all previous cases but also expands support for `determinate-nix-action` to a multitude of new CI environments.
If you'd like to give it a try, this example Actions workflow should do the trick:
```yaml title=".github/workflows/check.yaml"
jobs:
check:
name: Nix checks
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- name: Install Determinate Nix
uses: DeterminateSystems/determinate-nix-action@v3
- run: nix flake check -L
```
[act]: https://github.com/nektos/act
[announcement]: /blog/determinate-nix-action
[arc]: https://github.com/actions/actions-runner-controller
[determinate-nix-action]: https://github.com/DeterminateSystems/determinate-nix-action
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[docker]: https://docker.com
[kubernetes]: https://kubernetes.io
---
**Determinate Nix Action**
Published: May 28, 2025
URL: https://determinate.systems/blog/determinate-nix-action
Shortly after releasing [Determinate Nix Installer][nix-installer], we released [`nix-installer-action`][nix-installer-action], a GitHub Action for installing Nix using Determinate Nix Installer, our next-generation installer written in [Rust] with support for seamlessly uninstalling Nix and more.
This Action has successfully delivered Nix to millions of runners and thus served its purpose quite ably.
Since the [launch] of [Determinate Nix][det-nix], `nix-installer-action` has provided the option of installing Determinate Nix by applying `determinate: true` to your YAML configuration.
But `nix-installer-action` has an important drawback: its version isn't synced with the version of Determinate Nix, making it difficult to target Determinate Nix versions in CI runs.
And so today we're announcing that we've created a new Action, [`determinate-nix-action`][action] that installs Determinate Nix both by default and with a synced version.
That means that you can use, say, Determinate Nix version 3.6.1 by specifying this:
```yaml title="Targeted version"
- uses: DeterminateSystems/determinate-nix-action@v3.6.1
```
Here's a full example configuration:
```yaml title=".github/workflows/nix.yaml" {11,12,15}
on:
pull_request:
push:
branches: [main]
jobs:
build-nix-pkg:
name: Build Nix package
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- name: Build package using Determinate Nix
run: nix build
```
Note that the `permissions` block is necessary if you intend to use [private flakes][private-flakes] and [FlakeHub Cache][cache].
And as an alternative to using `v3`, you can pin to a specific release, such as [`@v3.6.1`][latest] (the most recent release as of this writing).
In terms of configuration, `determinate-nix-action` provides most of the same [parameters][configuration] that you're used to from `nix-installer-action`—such as `extra-conf` for providing custom Nix configuration—leaving behind only those that don't make sense in this new Action.
If you do opt to pin to a specific version of `determinate-nix-action`, we strongly recommend using tools like [Dependabot] and [Renovate] to automatically bump the version when necessary, lest you end up running out-of-date versions of Determinate Nix.
If you'd like to target specific versions of Determinate Nix, we do recommend switching to this new Action.
For more on why you should consider pinning your Actions, GitHub has a good [primer][security-guide].
And as with all Actions, you can pin `determinate-nix-action` to a specific Git hash, such as [`441b9e401ac050c38a07d8313748c5c2d17e8aff`][exact-hash]:
```yaml title=".github/workflows/nix.yaml"
- uses: DeterminateSystems/determinate-nix-action@441b9e401ac050c38a07d8313748c5c2d17e8aff
```
If you're currently using [`nix-installer-action`][nix-installer-action] to install Determinate Nix don't worry: nothing is changing on that end and you can still set `determinate: true` to install Determinate Nix:
```yaml title=".github/workflows/nix.yaml" {3}
- uses: DeterminateSystems/nix-installer-action
with:
determinate: true
```
[action]: https://github.com/DeterminateSystems/determinate-nix-action
[cache]: https://docs.determinate.systems/flakehub/cache
[configuration]: https://github.com/DeterminateSystems/determinate-nix-action?tab=readme-ov-file#%EF%B8%8F%EF%B8%8F-configuration
[dependabot]: https://github.com/dependabot
[det-nix]: https://docs.determinate.systems/determinate-nix
[exact-hash]: https://github.com/DeterminateSystems/determinate-nix-action/tree/441b9e401ac050c38a07d8313748c5c2d17e8aff
[latest]: https://github.com/DeterminateSystems/determinate-nix-action/releases/tag/v3.6.1
[launch]: /blog/announcing-determinate-nix
[nix-installer]: https://github.com/DeterminateSystems/nix-installer
[nix-installer-action]: https://github.com/DeterminateSystems/nix-installer-action
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[renovate]: https://github.com/renovatebot/renovate
[rust]: https://rust-lang.org
[security-guide]: https://docs.github.com/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions#using-third-party-actions
---
**Changelog: introducing lazy trees**
Published: May 15, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-352
[**Lazy trees**](#importance) have been one of the most hotly requested Nix features for quite some time.
They make Nix much more efficient in larger repositories, particularly in massive monorepos.
And so we're excited to announce that lazy trees have landed in [Determinate Nix][det-nix] version **3.5.2**, based on version [2.28.3][nix-version] of upstream Nix.
We're confident that this initial release of lazy trees should produce few issues for users.
But for the sake of caution, lazy trees is available solely on an **opt-in basis** for the time being.
To start using it today, first [install or upgrade](#upgrade-install) Determinate Nix and enable lazy trees in your custom Nix configuration:
```shell title="/etc/nix/nix.custom.conf"
lazy-trees = true
```
Our co-founder [Eelco Dolstra][eelco] currently has a [pull request open][lazy-trees-upstream-pr] to get lazy trees into [upstream Nix][upstream] and we hope to see that merged soon.
## Why lazy trees are important \{#importance}
In a nutshell, lazy trees provide **faster and less resource-intensive evaluation** in many standard usage scenarios.
For evaluations inside of the [Nixpkgs] repo, for example, we've frequently seen reductions in wall time of **3x or more** and reductions in disk usage of **20x or more**—and occasionally reductions far beyond even this.
With lazy trees enabled, Nix scopes file copying to what the specific expression demands—and nothing more.
More specifically, Nix uses a **virtual filesystem** to gather the necessary file state prior to copying anything to the [Nix store][store], "mounting" lazy source inputs to this virtual filesystem at `/nix/store/`.
This provides a much more parsimonious mode of operation that should quietly make just about everything in Nix better, from standard package builds to development environments to NixOS deployment and beyond.
If you'd like to see this in action, try evaluating something in Nixpkgs yourself.
[Install or upgrade Determinate Nix](#upgrade-install) and run this sequence of commands:
```shell title="Evaluate a derivation in Nixpkgs with lazy trees disabled"
# Clone Nixpkgs
git clone https://github.com/NixOS/nixpkgs
cd nixpkgs
# Evaluate the cowsay derivation with lazy trees disabled
time nix eval \
--no-eval-cache \
.#cowsay
# Make a meaningless change to the source tree
echo "" >> flake.nix
# Evaluate the cowsay derivation again
time nix eval \
--no-eval-cache \
.#cowsay
```
Adding an empty line to `flake.nix` (a meaningless change) invalidates the evaluation cache and forces Nix to copy the **entire source tree** to the [Nix store][store], otherwise the second evaluation would be faster.
Now the same sequence of steps with lazy trees:
```shell title="Evaluate a derivation in Nixpkgs with lazy trees enabled"
# Enable lazy trees
echo "lazy-trees = true" | sudo tee -a /etc/nix/nix.custom.conf
# Make a meaningless change to the source tree
echo "" >> flake.nix
# Evaluate the cowsay derivation
time nix eval \
--no-eval-cache \
.#cowsay
# Make a meaningless change to the source tree
echo "" >> flake.nix
# Evaluate the cowsay derivation again
time nix eval \
--no-eval-cache \
.#cowsay
```
You'll notice two things:
1. The evaluation should be substantially faster with lazy trees (if not, please let us know [via email][email] or [on Discord][discord]).
1. With lazy trees, you won't see any indication that Nix is copying the source tree to the Nix store.
That's because it's able to determine, via evaluation, that only the sources for the [`cowsay`][cowsay] package are necessary to achieve its ends.
Saving time is good.
Also good: using less disk space.
I'll provide an example.
First, let's evaluate something in Nixpkgs using a `./tmp` subdirectory inside the current directory as a fresh [Nix store][store]:
```shell title="Disk space usage with lazy trees disabled"
# Disable lazy trees
echo "lazy-trees = false" | sudo tee -a /etc/nix/nix.custom.conf
# Evaluate the cowsay package
nix eval \
--no-eval-cache \
--store ./tmp \
.#cowsay
# See how much disk space is used by the local Nix store
du -sh ./tmp
```
On my machine, this yields **304 megabytes**.
Now let's try the same thing with lazy trees enabled:
```shell title="Disk space usage with lazy trees enabled"
# Delete the local Nix store to start from scratch
sudo rm -rf ./tmp
# Enable lazy trees
echo "lazy-trees = true" | sudo tee -a /etc/nix/nix.custom.conf
# Evaluate the cowsay package
nix eval \
--no-eval-cache \
--store ./tmp \
.#cowsay
# See how much disk space is used by the local Nix store
du -sh ./tmp
# Delete the local Nix store to clean up
sudo rm -rf ./tmp
```
This yields **13 megabytes**.
That's over 23 times less disk space.
This specific number isn't generalizable across all situations, of course, but it does illustrate just how much more efficient Nix can become in a standard scenario when using lazy trees.
## CI comparisons
To illustrate some of these differences in a CI environment, I've set up a GitHub repo at [DeterminateSystems/lazy-trees-comparisons][comparisons-repo].
The [workflow configuration][comparisons-workflow] shows the sequences of steps involved (similar to the steps [above](#importance)).
As an example, check out [this CI run][comparisons-result], which evaluates the `stdenv` attribute of Nixpkgs on GitHub Actions' `ubuntu-latest` runner.
The evaluation time without lazy trees:
```shell title="Evaluation time without lazy trees"
real 0m10.787s
user 0m3.655s
sys 0m6.759s
```
Wall time without lazy trees: almost **11 seconds**.
Disk usage without lazy trees: **433 megabytes**.
Evaluation time with lazy trees:
```shell title="Evaluation time with lazy trees"
real 0m3.500s
user 0m1.762s
sys 0m1.701s
```
Wall time without lazy trees: **3.5 seconds**.
Disk usage with lazy trees: **11 megabytes**.
Again, a substantial difference.
This is, of course, just one result of one run, but it's consistent with [more general runs][comparisons-group] and we encourage you to make your own comparisons.
## New warning
When using Determinate Nix with lazy trees, you may see warnings like this:
```shell title="Inefficient copy warning"
warning: Performing inefficient double copy of path '«github:nixos/nixpkgs/0c0bf9c057382d5f6f63d54fd61f1abd5e1c2f63»/' to the store. This can typically be avoided by rewriting an attribute like `src = ./.` to `src = builtins.path { path = ./.; name = "source"; }`.
```
You'll see this in cases where you specify the root of a repo as the source for a derivation.
When you use `./.` as the source for a derivation, Nix copies your repo to the Nix store under the name `/nix/store/-source`, which is then copied *again* as `/nix/store/--source`.
Because of that, this is an inefficient way to specify your source:
```nix title="flake.nix"
{
myPackage = myDerivationFunction {
src = ./.;
};
}
```
This is a much more efficient way:
```nix title="flake.nix"
{
myPackage = myDerivationFunction {
src = builtins.path { path = ./.; name = "my-package-source"; };
};
}
```
The `name` field can be whatever you like.
In a flake, you can also use `self` instead of `./.`, as in:
```nix title="flake.nix"
{
myPackage = myDerivationFunction {
src = self;
};
}
```
Ideally, this wouldn't be necessary and using `./.` wouldn't produce any inefficiency.
But unfortunately, changing this behavior would constitute a backwards-incompatible change that we are highly averse to introducing.
## How to use lazy trees in GitHub Actions \{#actions}
If you'd like to try out lazy trees in [GitHub Actions][actions], you can use our [Determinate Nix Action][determinate-nix-action] and supply `lazy-trees = true` in your configuration:
```yaml title=".github/workflows/build-with-lazy-trees.yaml" {5,8}
steps:
- uses: actions/checkout@v4
- name: Install Determinate Nix with lazy trees enabled
uses: DeterminateSystems/determinate-nix-action@v3
with:
extra-conf:
lazy-trees = true
- name: Set up FlakeHub Cache
uses: DeterminateSystems/flakehub-cache-action@main
- name: Build default package output
run: nix build .
```
## How to upgrade or install Determinate Nix \{#upgrade-install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.5.2 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.5.2"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos-module].
To verify that you're on the most recent version of Determinate Nix at any time:
```shell title="Check Determinate Nix version"
determinate-nixd version
```
And again, once you *are* on the most recent version, you can enable lazy trees in your custom Nix configuration:
```shell title="/etc/nix/nix.custom.conf"
lazy-trees = true
```
On NixOS, enable the NixOS option:
```nix
nix.settings.lazy-trees = true;
```
## More to come
[Determinate Nix][det-nix] has introduced a number of improvements to Nix in recent months, including [JSON logging][json-logging] and [automatic fixes for hash mismatches][hash-mismatches], but we're confident that the lazy trees release presents the most consequential change to date because it establishes a new, qualitatively improved baseline for Nix performance.
But we are far from finished with improving Nix's performance.
In the coming months, we intend to ship more improvements to Nix's evaluation performance, including:
1. Improving evaluation caching
1. Bringing [parallel evaluation][parallel-eval] to more Nix operations
1. Providing multi-threaded unpacking of flakes, both flakes from tarballs (like those from [FlakeHub]) and those from source forges like GitHub.
Stay tuned!
[actions]: https://github.com/features/actions
[comparisons-workflow]: https://github.com/DeterminateSystems/lazy-trees-comparisons/blob/main/.github/workflows/comparison.yaml
[comparisons-repo]: https://github.com/DeterminateSystems/lazy-trees-comparisons
[comparisons-result]: https://github.com/DeterminateSystems/lazy-trees-comparisons/actions/runs/15051253370/job/42306364011
[comparisons-group]: https://github.com/DeterminateSystems/lazy-trees-comparisons/actions/runs/15051253370
[cowsay]: https://github.com/piuccio/cowsay
[det-nix]: https://docs.determinate.systems/determinate-nix
[determinate-nix-action]: https://github.com/DeterminateSystems/determinate-nix-action
[discord]: https://discord.com/invite/a4EcQQ8STr
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[eelco]: /people/eelco-dolstra
[email]: mailto:support@determinate.systems
[flakehub]: https://flakehub.com
[hash-mismatches]: /blog/changelog-determinate-nix-331#hash-mismatches
[json-logging]: /blog/changelog-determinate-nix-331#json-logging
[lazy-trees-upstream-pr]: https://github.com/NixOS/nix/pull/13225
[nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.28
[nixos-module]: https://docs.determinate.systems/guides/advanced-installation#nixos
[nixpkgs]: https://github.com/NixOS/nixpkgs
[parallel-eval]: /blog/parallel-nix-eval
[store]: https://zero-to-nix.com/concepts/nix-store
[upstream]: https://github.com/NixOS/nix
---
**Changelog: deprecating channels and indirect flake references**
Published: May 6, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-342
We're excited to announce the release of [Determinate Nix][det-nix] **3.4.2**, based on version [2.28.3][nix-version] of upstream Nix.
In our [last Determinate Nix release][last] we announced a bunch of fun new stuff, like [JSON logging][json-logging] and a new set of capabilities to handle [hash mismatches][hash-mismatches].
This time around, the vibe is a bit different, as we've opted to deprecate two capabilities in Nix:
* [Channels](#channels) as an abstraction for handling Nix inputs.
* [Indirect flake references](#indirect-flake-refs) in flakes.
## Deprecating channels \{#channels}
For a good chunk of Nix's life, [channels] were *the* way to do things.
If you needed to evaluate expressions from a Nix project, such as [Nixpkgs], you added a channel for it and periodically updated that channel to see newer Git commits.
Channels worked well enough but they always carried the grievous drawback that two systems looking at the same Nix code could "see" two different Nix expressions because their respective channels had been updated to different commits—which we believe goes firmly against the grain of Nix's worldview.
There's an additional problem that every user on a system gets their own channels (including the root user) and the root user's channels are inherited by all users (but not the other way around)—another source of persistent confusion.
For these and other reasons, we announced our intention to deprecate [channels] in [this issue][intent-channels].
This command, for example, would build different versions of [ponysay] from different commits of Nixpkgs on different machines *unless* those machines somehow managed to coordinate their channels:
```shell title="nix-build command involving an unpinned Nixpkgs"
nix-build --expr "(import {}).ponysay"
```
Although projects like [niv] helped the situation by providing a basic [pinning] mechanism, Nix deserved to have the [reproducibility] problem resolved at a more fundamental level—Nix deserved [flakes].
From the beginning of our life as a company, we at Determinate Systems [have believed][flakes-manifesto] that [flakes] are the future of Nix for a variety of reasons, but foremost because they [pin][pinning] your Nix dependencies at all times using a [`flake.lock`][lockfile] file, overcoming the core deficit of channels.
Now that [flakes have been stabilized][det-nix-launch] in Determinate Nix, we're ready to begin the process of **deprecating channels**.
In Determinate Nix 3.4.2, this means two things:
1. The [`nix-channel`][nix-channel] command now emits a deprecation warning any time you use it.
1. Both the `nix` CLI and the `nix-build` tool now emit warnings whenever you perform any actions related to channels.
This `nix-build` command, for example, still works:
```shell title="nix-build command emitting deprecation warning"
nix-build -I nixpkgs=channel:nixos-24.11 '' -A ponysay
```
The [`nix-build`][nix-build] command no longer performs any important functions now that the unified Nix CLI has been stabilized in Determinate Nix.
We still install it for you in case you still have `nix-build` invocations in your scripts but we intend to phase it out.
But because it uses the `nixos-24.11` channel, it emits this warning:
```shell title="nix-channel deprecation warning"
warning: Channels are deprecated in favor of flakes in Determinate Nix. Instead of 'channel:nixos-24.11', use 'https://nixos.org/channels/nixos-24.11/nixexprs.tar.xz'. See https://zero-to-nix.com for a guide to Nix flakes. For details and to offer feedback on the deprecation process, see: https://github.com/DeterminateSystems/nix-src/issues/34.
```
This unified `nix` CLI command also emits a similar deprecation warning because it uses a channel:
```shell title="Unified CLI command emitting deprecation warning"
nix build --file '' --nix-path nixpkgs=channel:nixos-24.11 ponysay
```
What we haven't yet deprecated is [lookup paths][lookup-paths] like ``.
So this command works without emitting a warning:
```shell title="Unified CLI command emitting no warning"
nix build --file '' ponysay
```
But it's important to note that it works because Determinate Nix's default settings contain an `extra-nix-path` setting that sets the value of `` to a flake reference:
```shell title="/etc/nix/nix.conf"
extra-nix-path = nixpkgs=flake:https://flakehub.com/f/DeterminateSystems/nixpkgs-weekly/*.tar.gz
```
Commands like this also still work because they implicitly refer to the lookup value set in `extra-nix-path`:
```shell
nix build --impure --expr '(import {}).cowsay'
```
While we don't necessarily encourage using the `` lookup path for *new* things, we know that many of you may have bits of Nix logic that use references like this and we want to continue to support this use case for now.
## Deprecating indirect flake references \{#indirect-flake-refs}
Nix with flakes has, from its inception, offered a [flake registry][registry] that enables you to map *indirect flake references* to real flake references, such as `nixpkgs` to `https://flakehub.com/f/NixOS/nixpkgs/*`.
With this mapping in place, these two commands would be equivalent:
```shell
# With full reference
nix flake show "https://flakehub.com/f/NixOS/nixpkgs/*"
# With implicit reference
nix flake show nixpkgs
```
The flake registry, which you can manipulate using the [`nix registry`][nix-registry] command, also enables you to use implicit references in flakes themselves.
In this example, the URL for the `nix` flake input maps to whatever is in the flake registry:
```nix title="flake.nix" {2}
{
inputs.nix.url = "nix/2.28.3";
outputs = { self, nix, ... }: {
# your outputs
};
}
```
This capability also enables you to forgo specifying inputs entirely:
```nix title="flake.nix" {2}
{
outputs = { self, nixpkgs, ... }: {
# your outputs
};
}
```
We at Determinate Systems think that the former use case for implicit flake references—providing "shortcut" CLI commands—is quite reasonable.
But we don't think that indirect references inside of flakes are reasonable.
In fact, we've seen this as an **ill-conceived misfeature** for years because it produces a hole in flakes' reproducibility guarantees, as the reference can resolve to different things depending on what's in the registry.
Although this *becomes* less of an issue when the indirect reference is resolved to a direct reference and written to the `flake.lock` file, indirect references still run against the spirit of flakes.
We announced our intention to deprecate indirect flake references in `flake.nix` files in [this issue][intent-flakerefs], and in Determinate Nix 3.4.2 we've made some changes to that effect.
This command still works...
```shell title="Indirect flake references on the command line"
nix run nixpkgs#cowsay -- Moooo
```
...but you will see a warning if you run commands against this `flake.nix` file...
```nix title="flake.nix"
{
outputs = { self, nixpkgs, ... }: {
# your outputs
};
}
```
...as well as this one:
```nix title="flake.nix"
{
inputs.nixpkgs.url = "nixpkgs";
outputs = { self, nixpkgs, ... }: {
# your outputs
};
}
```
The warning looks like this, providing a hint to how you can address the issue:
```shell title="Indirect flake reference warning"
warning: Flake input 'nixpkgs' uses the flake registry. Using the registry in flake inputs is deprecated in Determinate Nix. To make your flake future-proof, add the following to '/nix/store/96y77qcpivmvskagnd94hgqczrgcq2w6-source/flake.nix':
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
For more information, see: https://github.com/DeterminateSystems/nix-src/issues/37
```
In the future we intend to completely prohibit indirect flake references in flakes.
## How to upgrade or install \{#upgrade-install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.4.2 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.4.2"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos-module].
## Implications
Sometimes less really is more.
Here at Determinate Systems, we're on a mission to modernize Nix and make it not just suitable but rather indispensable for a wide range of enterprise use cases.
Sometimes, that will mean adding scintillating new features that expand the range of what Nix can do.
Other times, like today, it will mean engaging in some careful pruning.
[channels]: https://zero-to-nix.com/concepts/channels
[det-nix]: https://docs.determinate.systems/determinate-nix
[det-nix-launch]: /blog/determinate-nix-30
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[flakes]: https://zero-to-nix.com/concepts/flakes
[flakes-manifesto]: /blog/experimental-does-not-mean-unstable
[hash-mismatches]: /blog/changelog-determinate-nix-331#hash-mismatches
[intent-channels]: https://github.com/DeterminateSystems/nix-src/issues/34
[intent-flakerefs]: https://github.com/DeterminateSystems/nix-src/issues/37
[json-logging]: /blog/changelog-determinate-nix-331#json-logging
[last]: /blog/changelog-determinate-nix-331
[lockfile]: https://zero-to-nix.com/concepts/flakes#lockfile
[lookup-paths]: https://nix.dev/manual/nix/latest/language/constructs/lookup-path
[nix-build]: https://nix.dev/manual/nix/latest/command-ref/nix-build
[nix-channel]: https://nix.dev/manual/nix/latest/command-ref/nix-channel
[nix-registry]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-registry
[nix-version]: https://nix.dev/manual/nix/latest/release-notes/rl-2.28
[nixos-module]: https://github.com/determinatesystems/determinate?tab=readme-ov-file#installing-using-our-nix-flake
[nixpkgs]: https://github.com/NixOS/nixpkgs
[niv]: https://github.com/nmattia/niv
[pinning]: https://zero-to-nix.com/concepts/pinning
[ponysay]: https://github.com/erkin/ponysay
[registry]: https://zero-to-nix.com/concepts/flakes/#registries
[reproducibility]: https://zero-to-nix.com/concepts/reproducibility
---
**Changelog: JSON logging, a new experience around hash mismatches, and more**
Published: April 16, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-331
Logging provides a window into what a program is up to, and that window is never more crucial than when something has gone wrong and you need granular insight into what has happened and how you can fix it.
In Nix, logging is generally most important during the [realisation] process, and *ideally* Nix would provide world-class build error logs.
But historically this has *not* been seen as one of Nix's strengths.
We've set out to change that, and we're excited to announce that with the 3.3.1 release [Determinate Nix][det-nix] now supports [**JSON logging**](#json-logging) for a small subset of error logs (with a larger subset to come soon).
Adding this capability to Nix has also enabled us to dramatically upgrade the user experience around things like fixing [hash mismatches](#hash-mismatches).
Check out this video from Determinate Systems CEO and cofounder [Graham Christensen][graham] for a quick overview:
Once you've done that, read on for some more in-depth information.
## JSON logging
With Determinate Nix 3.3.1, Nix can now log some error messages as structured JSON.
We've added this because JSON is generally more straightforward to parse by external tools.
At this stage, Nix provides JSON logs in one specific case: when Nix encounters a [hash mismatch](#hash-mismatches) during a build.
In the future we plan to enable JSON logging in more places but we think this is a good place to start.
## Automatic fixes for hash mismatches \{#hash-mismatches}
Seasoned users of Nix are likely all-too familiar with this log output:
```shell title="Hash mismatch log output"
error: hash mismatch in fixed-output derivation '/nix/store/2ybj73ri0dc9j28yzqlp7r1l2lpxnsxj-my-package.drv':
specified: sha256-tmNn6/9jn/ansLI0QTfy6/fItRmBCOhb5AUOXbIWi24=
got: sha256-GzabkpW8L2UH4rzBkFdv61RoJl8DQlICqZvY9Ql1MAU=
```
This certainly *does* provide the information needed to fix the problem.
But in order to actually extract the correct hash—the information you really want here—you need to parse this log *as a string*.
You can do this with [grep] and other tools but let's face it—it isn't a whole lot of fun.
With JSON logging, however, Nix produces this instead:
```json title="Hash mismatch JSON output"
{
"v": "1",
"c": "HashMismatchResponseEventV1",
"drv": "/nix/store/2ybj73ri0dc9j28yzqlp7r1l2lpxnsxj-my-package.drv",
"bad": [
"sha256-tmNn6/9jn/ansLI0QTfy6/fItRmBCOhb5AUOXbIWi24="
],
"good": "sha256-GzabkpW8L2UH4rzBkFdv61RoJl8DQlICqZvY9Ql1MAU="
}
```
You can parse JSON logs with tools like [jq] instead of, say, grep with a set of gnarly regular expressions.
### Fix hashes using Determinate Nixd \{#determinate-nixd-hashes}
[Determinate Nixd][dnixd], which comes installed with [Determinate Nix][det-nix], enables you to automatically fix mismatched hashes:
```shell title="Apply hash fixes using Determinate Nixd"
determinate-nixd fix hashes
```
Let's say that you've just run `nix build` and gotten an error because, say, a vendor hash in a derivation building a [Go] application is wrong.
If you run the command above, Determinate Nixd will prompt you to ask if you'd like to fix hash mismatches for you.
If you say **yes**, Determinate Nixd fixes the relevant files in place; all you need to do is commit the changes and you're ready to go.
### Identify hash mismatches in GitHub Actions
While fixing hashes in your local development flows is nice, pretty much all of us forget to do this until it's too late and our continuous integration workflows are unhappy with us.
But with JSON logging now in Determinate Nix, we offer a solution to hash mismatches at that level as well.
If you'd like substantially improved hash mismatch *reporting*, all you have to do is use our [determinate-nix-action] for GitHub Actions.
```yaml title="Install Determinate Nix in GitHub Actions" {3}
- uses: DeterminateSystems/determinate-nix-action@v3
```
When you do that, you now get a GitHub-native notification rather than needing to dig through Nix error logs.
This screenshot shows an example:
Note that this is now surfaced directly in the Git diff UI—and in an attention-grabbing red color to boot.
### Automatic hash mismatches fixes in GitHub Actions
To finish drawing the owl, run `determinate-nixd fix hashes --auto-apply` on failure to fix pull requests automatically:
```yaml title="Automatically correct hash mismatches" {3-4,12,16-27}
jobs:
build:
permissions:
contents: write
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
- uses: DeterminateSystems/determinate-nix-action@v3
- run: nix flake check -L
- name: Fix hash mismatches
if: failure()
run: |
determinate-nixd fix hashes --auto-apply
if ! git diff --quiet; then
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add .
git commit -m "[dependabot skip] Fix hashes"
git push
fi
```
## Ergonomic fixes
In addition to [JSON logging](#json-logging) and the new features that it has opened up, we've made some smaller improvements that we hope you'll like as well.
### Better suggestions for untracked files \{#untracked-files}
When you use [flakes], Nix needs to work closely with Git to function properly.
Sometimes there are untracked changes in your repository that keep Nix from working as expected.
For years, Nix has emitted this log message when that happens:
```shell title="Previous error message for untracked changes"
error: path '/nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source/flake.nix' does not exist
```
That's... pretty opaque.
It's not clear why that file existing is somehow important and it doesn't tell you how to fix the problem.
With Determinate Nix 3.3.1, you now see an error message like this instead:
```shell title="New error message for untracked changes"
error: Path 'flake.nix' in the repository "/home/justme/my-project" is not tracked by Git.
To make it visible to Nix, run:
git -C "/home/justme/my-project" add "flake.nix"
```
Nix now provides not just a better error message but also actionable information that you can use to remedy the situation.
This has been a paper cut for Nix users for *years* and we're really happy to put it behind us for good.
### `nix profile install` is now `nix profile add` \{#nix-profile-add}
The [`nix profile`][nix-profile] command has an [`install`][nix-profile-install] subcommand that you can use to add a package to the current Nix profile.
This command works as expected but we've always felt like "install" isn't really the right verb—and many of our users have shared that sentiment.
In Determinate Nix 3.3.1, this has been replaced with an `add` subcommand:
```shell title="The new command to add a package to the current profile"
nix profile add nixpkgs#ponysay
```
Running `nix profile install` still works as an alias, but we do recommend switching any scripts over to the `add` subcommand now.
Like the [untracked files error message](#untracked-files), this isn't an earth-shattering change but it still feels good to bring Nix more in line with users' expectations.
## How to upgrade or install \{#upgrade-install}
If you already have [Determinate Nix][det-nix] installed, you can upgrade to 3.3.1 with one [Determinate Nixd][dnixd] command:
```shell title="Upgrade command for version 3.3.1"
sudo determinate-nixd upgrade
```
If you don't yet have Determinate Nix installed, you can install it on **macOS** using our graphical installer:
On Linux:
```shell title="Install Determinate Nix on Linux"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
On NixOS, we recommend using our [dedicated NixOS module][nixos-module].
## More to come
Nix now has the necessary code paths to output JSON logs wherever they may be needed.
We've started modestly with adding such logs, but even this modest beginning provides some powerful new workflows for Nix.
We have some ideas for other areas where JSON logs can open up new paths in Nix and we'll be exploring those soon.
On top of that, we've provided some [ergonomic changes](#ergonomic-fixes) that address small but long-standing and substantial user experience issues in Nix.
And we plan to tackle a few more issues along these lines in the coming weeks.
That's it for now!
Stay tuned here on the blog for more advancements in the whole experience around using Nix.
[det-nix]: https://docs.determinate.systems/determinate-nix
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[flakes]: https://zero-to-nix.com/concepts/flakes
[go]: https://golang.org
[graham]: /people/graham-christensen
[grep]: https://www.gnu.org/software/grep/manual/grep.html
[jq]: https://jqlang.org
[determinate-nix-action]: https://github.com/DeterminateSystems/determinate-nix-action
[nix-profile]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-profile
[nix-profile-install]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-profile-install
[nixos-module]: https://github.com/determinatesystems/determinate?tab=readme-ov-file#installing-using-our-nix-flake
[realisation]: https://zero-to-nix.com/concepts/realisation
---
**Best practices for Nix at work**
Published: March 24, 2025
URL: https://determinate.systems/blog/best-practices-for-nix-at-work
Last week, we at Determinate Systems held our first webinar, [Best practices for Nix at work][past], presented by [yours truly][me].
The session was a lot of fun, there were some great questions, and we're really excited to continue to do more.
It was so fun, in fact, that we'll be doing another one next week, on **Thursday, March 27th at 1 pm GMT-3**, titled [Up and running with FlakeHub Cache][upcoming].
It will cover some conceptual basics around binary caching in Nix, what sets [FlakeHub Cache][cache] apart from other offerings, and how to best take advantage of FlakeHub Cache's unique feature set.
I hope to see you there!
In this post, I'd like to share some of the best practices that I covered in our [initial webinar][past].
As in the talk, I've split those into three sections:
- [Best practices for using flakes](#flakes)
- [Security best practices for Nix](#security)
- [Best practices for organizational adoption](#adoption)
## Best practices for using flakes \{#flakes}
At Determinate Systems, we've made it [abundantly clear][stable] that we believe that [flakes] are the future of Nix.
So I won't cover *why* you should use flakes here, but I will lay out some best practices for *how* you should use flakes based on our deep experience as a company the last few years.
### Use SemVer \{#semver}
On their own, flakes don't have a concept of [semantic versioning][semver] (SemVer).
That's because most current Nix users pull flakes, such as [Nixpkgs], from source forges like [GitHub].
And since those source forges don't themselves have any concept of semantic versioning, at least when it comes to serving tarballs, Nix doesn't have the requisite information to work with.
Other sources for flakes, like the local filesystem, also don't have the appropriate "hooks" that Nix would need to support SemVer.
We at Determinate Systems have added SemVer to flakes with [FlakeHub], and unsurprisingly we do recommend using it.
SemVer in Nix can apply not just to CLI tools and standard packages, but also to things like running services.
Why not publish version 1.4.0 of your authentication service or 0.2.5 of your main web application?
You can even use Nix adoption as a reason to finally SemVer-ify your stack.
With SemVer, your Nix updates can become much smoother.
Rather than updating a monolithic dependency like [Nixpkgs] every however-many months, you can make your updates much more granular, handling any issues with discontinuity in a much more gradual way.
And if a flake outputs many different versioned things?
See more about that [below](#many-flakes).
### Split things up into as many flakes as you need \{#many-flakes}
In the past, the Nix community has generally gravitated toward a usage model that involves having few dependencies.
We see this centralizing tendency in many corners and frequently encounter organizations that depend only on [Nixpkgs] in all of their projects.
But things don't need to be so centralized.
[Flakes] are a great mechanism for dividing Nix things up into smaller units, and we built [FlakeHub] to make flakes "cheap"—cattle, not pets, if you will.
We built the [`flakehub-push`][flakehub-push] Action to make this trivial.
And don't forget that you can put several flakes in a single repository, making this model friendly to large repos or even monorepos.
So *how many* flakes do you need?
There's no hard-and-fast rule here, but in general we recommend publishing **one flake per versioned thing**, where a "thing" could be, for example, a versioned package like a CLI tool or [NixOS] configurations for a service or a series of [Home Manager][hm] configurations.
I might even venture to say that when in doubt, just create another flake.
You can always consolidate into larger flakes later if need be.
### Update your inputs often \{#update-inputs}
Keeping your inputs fresh can help make your updates smoother.
It can make the difference between fixing a series of small things incrementally over time and spending an afternoon trying to disentangle a mess where you're not sure what is breaking what.
We have two tools that can help with this:
* Our [Flake Checker Action][flake-checker] helps you keep your [Nixpkgs] inputs up to date and either warns you or straight-up fails your CI runs if Nixpkgs is more than 30 days old (or a different number of days if you wish).
* Our [update-flake-lock] Action automatically updates your flake inputs on your preferred cron schedule and creates a pull request.
### Avoid flake helper libraries if possible
While we do encourage splitting things up into flakes, we generally recommend not using flake helper libraries like [`flake-utils`][flake-utils] and [`flake-parts`][flake-parts].
There is nothing intrinsically *wrong* with these libraries—they're made by Nix veterans and they're well built and maintained—but any time you introduce a flake input into your dependency tree, it's then an input to anyone depending on your flake (and anyone depending on that, and so on).
And so you should strive to eliminate unnecessary inputs when possible, and we think that these are generally not necessary.
Let's start with `flake-utils`, which provides some helpers for system-specific flake outputs.
A worthy goal, certainly, but a few lines of Nix code will get you the same thing.
At Determinate Systems, for example, we have a few lines of boilerplate that we include in every flake to handle system-specific outputs.
The lines look something like this:
```nix title="flake.nix"
let
supportedSystems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ];
forEachSupportedSystem = f: nixpkgs.lib.genAttrs supportedSystems (system: f {
pkgs = import nixpkgs { inherit system; };
});
in { ... }
```
Then we can use that for things like package outputs:
```nix title="flake.nix"
{
packages = forEachSupportedSystem ({ pkgs }: {
client = ...;
server = ...;
});
}
```
`flake-parts` has more ambitious goals, striving to make flakes more module driven, but at the cost of, to my eyes at least, making flakes [more verbose][flake-parts-example] and convoluted.
At Determinate Systems, we always prefer plain old Nix code or Nixpkgs `lib` functions and have always managed without abstractions like this.
If you truly need `flake-utils` or `flake-parts`, please don't hesitate to use them.
Just be aware of the cost that flake helpers like this incur and judge accordingly.
If you find yourself relying on them in ways that can be readily replaced by a few lines of copy/pasted code, we say go with that.
## Security best practices for Nix \{#security}
One of the great promises of Nix is that it's supposed to make building and deploying things more secure by eliminating some common sources of non-reproducibility.
But Nix isn't magic here and there are still some basic precautions you should follow.
### Keep secrets out of the Nix store \{#secrets}
Always keep in mind that the [Nix store][store] isn't a typical filesystem for running ad-hoc operations like reading, writing, copying, and deleting files.
Instead, it's basically an instrument that Nix uses to function properly, and when you write something to the Nix store, it can be quite non-trivial to get it out.
And so you should engage in whatever practices necessary to prevent yourself from writing secrets to any Nix store—especially if that Nix store happens to be a cache like [FlakeHub Cache][cache].
One common practice in the Nix community is to use tools like [agenix] and [sops-nix] for managing secrets.
These are really solid tools and we think that they're fine for smaller organizations or deploying a small set of NixOS machines for personal use.
But for larger organizations, we recommend going beyond this and avoiding static secrets entirely—or at least whenever possible.
That could mean using [Vault] or an analogous service.
### Follow the principle of least access \{#least-access}
In larger organizations, we recommend a general policy of keeping your Nix things private and then granting access to them only when necessary.
For flakes, that means keeping your flakes [private][private-flakes] unless you absolutely need to distribute something to the Nix-using public.
This ensures that only people inside your org can run operations like `nix build` or even `nix flake show` against those flakes.
For caching, this means avoid public caches like [cache.nixos.org][cno] whenever possible and using trusted private caches like [FlakeHub Cache][cache] instead.
Public caches have lots and lots of good stuff in them, of course, but they're massive and the security and review practices behind them are not always air tight.
When you pull from them to build production systems, you're working with a porous trust model.
We built FlakeHub Cache as a strictly private cache with no public access whatsoever to fill this precise gap in traditional approaches to Nix caching.
## Best practices for organizational adoption \{#adoption}
Regardless of how wise the best practices mentioned above may be, they don't mean much if you can't get your organization up and running with Nix.
While there is no one correct path to Nix adoption, we do have a few rules of thumb worth drawn from our experience working with organizations of many sizes.
### Spread knowledge
While we and many others have been working hard to make Nix more straightforward to understand and use, it can be a challenge to get up to speed even with some conceptual basics like knowing what Nix *even is* and what benefits it can provide.
The more knowledge about Nix that you can spread within your organization, the better.
We created [Zero to Nix][z2n] as a flake-first documentation source for beginners, and we do strongly recommend passing that along to your coworkers.
There are also some great official resources, such as [nix.dev].
### Adopt a gradual approach \{#gradual}
Bringing Nix into any organization can be disruptive in a variety of ways.
But because Nix can transform so many things, there are lots of ways to gently expand usage.
Let's say that you're using Nix on just one team and only for [development environments][envs].
Just one team can be enough to demonstrates gains—faster CI runs, more synchronized environments, fewer "works on my machine" mishaps—significant enough to compel other teams to take action.
So you can talk a second team into using Nix for development environments.
And talk the first team into using Nix for packaging their tools and distributing them to other teams.
And maybe talk a third team into converting their `Dockerfile`-driven services into [NixOS] services.
Or you can convert all of the Python stuff in your org over to using Nix and then talk the PHP folks into doing the same.
And so on.
Ideally, from our perspective, you would wake up one morning with your entire org safely and happily under the Nix umbrella, but we know that things don't work that way.
Fortunately, there are many paths to expanding internal adoption in the meantime.
## See you soon?
This set of best practices isn't super systematic but it does provide a taste of what we went over in the webinar and what kind of content you can expect in the future there.
We hope to see you [at the next one][upcoming] on FlakeHub Cache this week!
[agenix]: https://github.com/ryantm/agenix
[cache]: https://flakehub.com/cache
[cno]: https://cache.nixos.org
[envs]: https://zero-to-nix.com/concepts/dev-env
[flake-checker]: https://github.com/DeterminateSystems/flake-checker-action
[flake-parts]: https://flake.parts
[flake-parts-example]: https://flake.parts/getting-started.html#existing-flake
[flake-utils]: https://github.com/numtide/flake-utils
[flakehub]: https://flakehub.com
[flakehub-push]: https://github.com/DeterminateSystems/flake-checker
[flakes]: https://zero-to-nix.com/concepts/flakes
[github]: https://github.com
[hm]: https://flakehub.com/flake/nix-community/home-manager
[me]: /people/luc-perkins
[nixos]: https://github.com/NixOS/nixpkgs/tree/master/nixos
[nixpkgs]: https://github.com/NixOS/nixpkgs
[nix.dev]: https://nix.dev
[past]: https://lu.ma/87xuszxp
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[semver]: https://docs.determinate.systems/flakehub/concepts/semver
[sops-nix]: https://github.com/Mic92/sops-nix
[stable]: /blog/experimental-does-not-mean-unstable
[store]: https://zero-to-nix.com/concepts/nix-store
[upcoming]: https://lu.ma/ouooinmw
[update-flake-lock]: https://github.com/DeterminateSystems/update-flake-lock
[vault]: https://vaultproject.io
[z2n]: https://zero-to-nix.com
---
**Determinate Nix 3.0**
Published: March 5, 2025
URL: https://determinate.systems/blog/determinate-nix-30
Today, we're thrilled to announce the release of [Determinate Nix 3.0][det-nix].
Determinate Nix is our professionally maintained, security-focused distribution of [Nix] designed for organizations that cannot compromise on security, compliance, or performance.
The version number matters.
The Nix project has long intended to release version 3.0 when [flakes] are stable.
With Determinate Nix 3.0, we've fulfilled that promise by offering a **formal stability guarantee for flakes**, making them production ready today.
## Why flake stability matters
Flake stability is essential for organizations developing mission-critical systems.
Although flakes enhance the core features of Nix—reproducible environments, predictable builds, composable configurations—by enforcing [pinned][pinning] dependencies, we continue to see deep uncertainty in the ecosystem due to flakes' experimental status in upstream Nix.
Without such a guarantee, organizations have expressed concerns about unpredictable breaking changes that could disrupt CI/CD pipelines, developer workflows, and production deployments.
Our flake stability guarantee mitigates this risk, ensuring that your flake-based configurations function reliably through every update and enabling you to confidently adopt flakes for your most important projects today.
## Not a fork, a future-proof foundation
Determinate Nix 3.0 isn't a fork of Nix—it's a downstream distribution that ensures full compatibility with the broader Nix ecosystem while incorporating the governance, security features, and performance optimizations necessary for mission-critical deployments.
## Building a better Nix
Determinate Nix 3.0 represents a fundamental shift in how we deliver Nix to organizations with mission-critical requirements.
We don't just package upstream Nix—we build it ourselves from source in our secure, [SLSA build level 3][slsa-3], controlled infrastructure.
This independent build process is crucial for the trust, security, and stability that regulated environments demand.
By maintaining our distribution, we can innovate at the pace our customers need without being constrained by upstream development cycles.
This means that Determinate Nix customers will soon have immediate access to critical features like:
- [**Parallel evaluation**][parallel-eval-pr] of Nix expressions, which can deliver 3-4x faster performance for complex configurations.
- [**Lazy trees**][lazytrees-pr], which can dramatically improve evaluation speed for large source repositories.
Our approach combines the best of both worlds: Determinate Nix customers benefit from cutting-edge capabilities today while we maintain compatibility with the broader Nix ecosystem and actively contribute our improvements to upstream Nix, where the community ultimately determines their inclusion.
Choosing Determinate Nix gives you priority access to innovations that address real-world needs while ensuring seamless integration with existing systems.
## Security that meets modern standards
Security is central to every aspect of Determinate Nix and the entire [Determinate platform][determinate].
- [**SOC 2 Type II certified**][soc2]: Our infrastructure meets rigorous compliance standards.
- **Zero-trust security model**: Federated authentication combined with fine-grained access controls with federated authentication.
- **Modern authentication**: Shift from static secrets to secure, policy-driven identities through Identity and Access Management (IAM) roles and identity providers such as [GitHub][github-auth] and [Microsoft Entra][entra-auth].
- **Corporate network integration**: Automatic certificate handling for [ZScaler], [Fortinet], and other security platforms.
- **Defined security response**: We provide an SLA for vulnerability management, ensuring prompt and predictable security updates.
- **Controlled build environments**: We build every component in our security-certified infrastructure, with critical components receiving additional signing and notarization.
## Stability you can rely on
When your business relies on dependable software delivery, you require predictability and consistency:
- **Flake stability guarantee**: While flakes remain experimental in upstream Nix, we provide a formal stability guarantee to ensure your workflows remain functional.
- **Rigorous release process**: Every change undergoes comprehensive security, performance, and compatibility validation.
- **Customer roadmap**: Transparent decision making and predictable release cycles.
## Deployment options for every need
From bare metal to virtual machines, cloud to edge, Determinate Nix 3.0 delivers reproducible builds everywhere:
- [**First-class macOS support**][nix-darwin-support]: We provide full [MDM] integration, including partnerships with leading MDM providers like [jamf], and ensure seamless macOS upgrades across your Apple ecosystem.
- **Performance optimizations**: Ship code faster with improved build times and streamlined workflows.
- **Intelligent resource management**: Automated garbage collection monitors system conditions to optimize maintenance.
## Why organizations choose Determinate Nix
From financial services to healthcare to critical infrastructure, teams choose Determinate Nix when they require:
- Predictable feature implementation with clear timelines for new capabilities.
- Security and compliance that meet and exceed industry requirements.
- Dramatically faster build times that slash evaluation overhead and accelerate development cycles by 3-4x in typical workloads.
- Operational workflows that scale from small projects to global deployments.
Determinate Nix 3.0 transforms Nix from a powerful tool into a trusted platform that delivers the governance, performance, and stability that serious production environments demand.
## Get started with Determinate Nix 3.0 today!
Ready to experience Determinate Nix 3.0?
### Installing Determinate Nix 3.0
Installation is straightforward whether you're deploying across an organization or installing on your workstation.
Our [comprehensive documentation site][determinate] provides tailored installation instructions for most environments.
### Upgrading to Determinate Nix 3.0
Follow these straightforward upgrade instructions for your environment.
#### For macOS users
Download the universal package and run it.
When the installation is complete, open a terminal and run:
```shell title="Check Nix version"
nix --version
```
You should see `nix (Determinate Nix 3.0.0) 2.26.3`.
#### For NixOS users
If you're already using the [Determinate flake][determinate-flake] in your [NixOS] configuration, update the flake:
Update the inputs section `flake.nix` to reference the latest Determinate version:
```nix title="flake.nix"
{
inputs.determinate.url = "https://flakehub.com/f/DeterminateSystems/determinate/*";
}
```
Run a system rebuild to apply the changes:
```shell title="Upgrade NixOS"
sudo nixos-rebuild switch --flake .
```
Log in to [FlakeHub]:
```shell title="Log in with determinate-nixd"
determinate-nixd login
```
Verify your upgrade:
```shell title="Check Nix version"
nix --version
```
You should see `nix (Determinate Nix 3.0.0) 2.26.3`.
#### For Linux users
First, download the appropriate `determinate-nixd` for your platform:
#### Linux (x86_64)
```shell title="Download Determinate Nix for Linux (x86_64)"
curl -sSLfo determinate-nixd https://install.determinate.systems/determinate-nixd/stable/x86_64-linux
```
#### Linux (aarch64)
```shell title="Download Determinate Nix for Linux (aarch64)"
curl -sSLfo determinate-nixd https://install.determinate.systems/determinate-nixd/stable/aarch64-linux
```
Make the file executable and run the upgrade:
```shell title="Upgrade Determinate Nix"
chmod +x ./determinate-nixd
sudo ./determinate-nixd upgrade
```
Log in to [FlakeHub]:
```shell title="Log in with determinate-nixd"
determinate-nixd login
```
The upgrade process will:
- Preserve all your existing Nix configurations
- Update to Determinate Nix 3.0 components
- Restart the Nix daemon with the new version
Verify your upgrade:
```shell title="Check Nix version"
nix --version
```
You should see `nix (Determinate Nix 3.0.0) 2.26.3`.
You can now remove the temporary installation file:
```shell title="Remove temporary installation file"
rm ./determinate-nixd
```
For [FlakeHub] teams, your authentication and access to [private flakes][private-flakes] and [caches][cache] will remain intact throughout the upgrade process.
If you encounter any issues during the upgrade or have additional questions, enterprise customers should contact their dedicated account representative directly.
All users are also welcome to [join our Discord server][discord] for community support.
## Contact us to transform your Nix experience
Ready to transform the way your team builds critical software?
Experience Determinate Nix 3.0 today and leave behind dependency nightmares, security headaches, and performance bottlenecks.
Email [hello@determinate.systems](mailto:hello@determinate.systems) to discuss how we can empower your team with a robust foundation that enables you to focus on what matters most: creating exceptional software that drives your business forward.
[det-nix]: https://docs.determinate.systems/determinate-nix
[nix]: https://zero-to-nix.com/concepts/nix
[flakehub]: https://flakehub.com
[flakes]: https://zero-to-nix.com/concepts/flakes
[slsa-3]: https://slsa.dev/spec/v1.0/levels#build-l3
[determinate]: https://docs.determinate.systems
[entra-auth]: https://learn.microsoft.com/entra/identity/authentication/overview-authentication
[github-auth]: https://docs.github.com/authentication
[lazytrees-pr]: https://github.com/NixOS/nix/pull/6530
[parallel-eval-pr]: https://github.com/NixOS/nix/pull/10938
[soc2]: https://trust.determinate.systems/
[zscaler]: https://www.zscaler.com/
[fortinet]: https://www.fortinet.com/
[mdm]: https://docs.determinate.systems/guides/mdm
[nix-darwin-support]: /blog/nix-darwin-updates/
[jamf]: https://www.jamf.com/
[determinate-flake]: https://flakehub.com/flake/DeterminateSystems/determinate?view=usage
[nixos]: https://zero-to-nix.com/concepts/nixos/
[discord]: https://discord.gg/invite/a4EcQQ8STr
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[cache]: https://docs.determinate.systems/flakehub/cache
[pinning]: https://zero-to-nix.com/concepts/pinning
---
**Fetch artifacts directly from FlakeHub Cache using `fh fetch`**
Published: February 26, 2025
URL: https://determinate.systems/blog/fh-fetch
Building stuff with Nix is extremely cool but do you know what's even cooler?
Letting another machine build it so that all you have to do is fetch the result.
If that sounds appealing then you're in luck because we recently added a `fetch` command to [fh], the CLI for [FlakeHub], that enables you to fetch [closures] directly from [FlakeHub Cache][cache].
Here's an example fetch command that should work on your system if you have [fh] 0.1.22 or later installed:
```shell title="Fetch a package from FlakeHub Cache"
NIX_SYSTEM=$(nix eval --impure --expr 'builtins.currentSystem' --raw)
fh fetch "DeterminateSystems/flake-iter/*#packages.${NIX_SYSTEM}.default" ./flake-iter
```
This pulls a CLI tool called [flake-iter] directly from FlakeHub Cache, puts it in the [Nix store][store], and writes a symlink to the `./flake-iter` path.
With the flake-iter package fetched, you can run the CLI:
```shell title="Run the fetched CLI"
./flake-iter/bin/flake-iter --help
```
No `nix build` command, no [store path evaluation][store-paths-post], just a [flake output reference][flake-ref] and you're all set.
## How to use it
In order to use the `fh fetch` command, you need to do a few things:
1. Get started with [Determinate] following the onboarding guide for either [organizations] or [individuals].
This includes [signing up][signup] for a paid plan with FlakeHub, which provides access to FlakeHub Cache.
1. [Install fh][install] version [0.1.22][v0.1.22] or later.
1. Start [pushing things][using] to FlakeHub Cache.
1. [Log in][login] to FlakeHub.
## Example use case: fetching OCI images \{#oci-images}
Up at the top, we saw an example of using `fh fetch` to pull a CLI tool from FlakeHub Cache.
Another great use case is pulling [Open Container Initiative][oci] (OCI) container images, such as [Docker] or [Podman] images.
I'm a macOS-only dev, which means that working with containers in Nix can be challenging because I can't actually *build* Linux containers locally without resorting to a remote builder.
I know how to set up a remote builder because I'm a long-time Nix user, but if you're in an organization with many macOS folks, it may be much more straightforward to build images in CI and have people fetch them.
Here's an example command that does just that:
```shell title="Fetch an OCI image directly from FlakeHub Cache"
fh fetch \
"DeterminateSystems/fh-fetch-example/*#dockerImages.x86_64-linux.server" ./oci-image
```
Since FlakeHub Caches are private, this command *will fail* for anyone outside of the `DeterminateSystems` org.
Fetching only pulls from FlakeHub Cache—it never builds or even evaluates the store path—and without access to that specific flake, you're out of luck.
And so even though both the [repo][example-repo] and the [flake on FlakeHub][example-flake] are public, the fetchable output is not (it would also be inaccessible, of course, if the repo or flake were [private][private-flakes]).
But since *I* am allowed to fetch, the command worked and I can now load the image into [Docker]:
```shell title="Load the Docker image from the target link"
docker load < ./oci-image
```
That provides this output:
```shell title="Log output with the container's name and tag"
Loaded image: web-server:5mll9s6m7m7pjha0k2krzq8kvjn111i3
```
And now I can run the image:
```shell title="Run the recently loaded image"
docker run \
--publish 8080:8080 \
web-server:lpr4bbf6v5yfzn1v1cfvlr69vbcr0iqh
```
Success!
Now I as a macOS user have a brand new option for working with Nix-built OCI images.
This is one use case among many for `fh fetch`, so be on the lookout for Nix-build things that you may need to securely distribute inside your own org.
## The FlakeHub platform branches out
With `fh fetch`, FlakeHub is beginning to transform into a powerful platform not just for secure access to [flakes] and secure binary caching but also as a **distribution mechanism** for things that you've built using Nix.
The ability to build and cache in CI—potentially from [private flakes][private-flakes]—and then provide your teams with secure access to built [closures] is genuinely novel in the Nix ecosystem.
How you use these distribution channels as a building block inside your organization is up to you, of course, but we're excited to see the directions that you take these novel capabilities.
[cache]: https://docs.determinate.systems/flakehub/cache
[closures]: https://zero-to-nix.com/concepts/closures
[determinate]: https://docs.determinate.systems
[docker]: https://docker.com
[example-flake]: https://flakehub.com/flake/DeterminateSystems/fh-fetch-example
[example-repo]: https://github.com/DeterminateSystems/fh-fetch-example
[fh]: https://flakehub.com/flake/DeterminateSystems/fh
[flake-iter]: https://github.com/DeterminateSystems/flake-iter
[flake-ref]: https://zero-to-nix.com/concepts/flakes/#references
[flakehub]: https://flakehub.com
[flakes]: https://zero-to-nix.com/concepts/flakes
[individuals]: https://docs.determinate.systems/getting-started/individuals
[install]: https://docs.determinate.systems/flakehub/cli/#installation
[login]: https://docs.determinate.systems/flakehub/cli/#login
[oci]: https://opencontainers.org
[organizations]: https://docs.determinate.systems/getting-started/organizations
[podman]: https://podman.io
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[signup]: https://flakehub.com/signup
[store]: https://zero-to-nix.com/concepts/nix-store
[store-paths-post]: /blog/resolved-store-paths
[using]: https://docs.determinate.systems/flakehub/cache/#using-flakehub-cache
[v0.1.22]: https://flakehub.com/flake/DeterminateSystems/fh/0.1.22
---
**Introducing Determinate AMIs for NixOS**
Published: February 24, 2025
URL: https://determinate.systems/blog/nixos-amis
Today, we're excited to announce that Determinate Systems now offers [Amazon Machine Images][amis] (AMIs) for [NixOS] that include [Determinate Nix][det-nix].
Our AMIs are available for both AMD64 Linux (`x86_64-linux` in Nix terms) and ARM64 Linux (`aarch64-linux`) and you can see the code behind them in our [nixos-amis] repo.
What sets these apart from other NixOS AMIs is that they seamlessly interoperate with [FlakeHub] and thus offer much speedier NixOS deployments, even inside of [autoscaling groups][autoscaling].
Interoperation with FlakeHub is provided by [Determinate Nix][det-nix] and [fh]:
* [Determinate Nix][det-nix], our validated and secure [Nix] for enterprises.
Determinate Nix includes [Determinate Nixd][dnixd], a utility that enables you to log in to [FlakeHub] using [AWS Secure Token Service][sts] (STS) using one command:
```shell title="Log in to FlakeHub from AWS with a single command"
determinate-nixd login aws
```
* [fh], the CLI for [FlakeHub].
You can use fh for things like [applying][fh-apply] NixOS configurations uploaded to [FlakeHub Cache][cache].
Here's an example:
```shell title="Apply a NixOS configuration to your AMI with two commands"
determinate-nixd login aws
fh apply nixos \
"my-org/my-flake/*#nixosConfigurations.my-nixos-configuration"
```
In this case, fh would pull the target NixOS [closure] from FlakeHub Cache without even needing to [evaluate the store path][store-paths].
We see this as a major upgrade over existing approaches to NixOS deployment, and you can read more about it in a [recent blog post][resolved-paths].
## Using Terraform or OpenTofu
While there are lots of ways to deploy AMIs to [EC2], we suspect that users are likely to gravitate toward [Infrastructure as Code][iac] (IAC) tools like [Terraform] and [OpenTofu].
Here's an example declaration of an [`aws_ami`][aws-ami-resource] data resource that uses our AMD64 Linux AMI:
```hcl title="Terraform/OpenTofu configuration"
data "aws_ami" "detsys_nixos" {
most_recent = true
# Determinate Systems' AMI owner ID
owners = ["535002876703"]
# Our AMIs are currently under the epoch-1 version
filter {
name = "name"
values = ["determinate/nixos/epoch-1/*"]
}
# AMD64 Linux
filter {
name = "architecture"
values = ["x86_64"]
}
}
```
With that AMI declared, you can create an [EC2] instance that applies a [NixOS] configuration upon launch:
```hcl title="An EC2 instance with an applied NixOS configuration" {2,6-7}
resource "aws_instance" "ethercalc-server" {
ami = data.aws_ami.detsys_nixos.id
instance_type = "t3.micro"
user_data = <
## What you should change
If you're currently using Determinate but not nix-darwin, you can start using nix-darwin at any time.
Just make sure to set `nix.enable = false` at the top level of your nix-darwin configuration.
If you're currently using Determinate with the [`determinate` module][determinate-flake] for nix-darwin, follow these steps:
- Upgrade both the `determinate` and the `nix-darwin` input in your configuration using `nix flake update`.
- Run `darwin-rebuild` for your configuration.
This should *fail*.
- After that fails, set `nix.enable = false;` and remove the Determinate nix-darwin module from your configuration.
- If you have Nix configuration you're currently setting with nix-darwin, add that to the `/etc/nix/nix.custom.conf` configuration file instead.
And that's it!
Nix is now managed by Determinate, while nix-darwin handles your macOS system configuration.
In terms of nix-darwin release compatibility, this approach works on both the 24.11 release branch as well as the rolling `master` branch.
If you're not yet using Determinate on macOS, you can get started now:
## Implications
Letting Determinate manage Nix while letting nix-darwin manage macOS system configuration plays much better to the strengths of both tools.
And beyond the benefit to Determinate users, providing nix-darwin users with an opt-out switch like this means that other tools and platforms are also free to manage Nix on macOS.
More configurability, after all, is the Nix way.
[announcement]: /blog/announcing-determinate-nix
[determinate]: https://docs.determinate.systems
[determinate-flake]: https://github.com/DeterminateSystems/determinate
[emily]: https://github.com/emilazy
[gui]: /blog/graphical-nix-installer
[macos]: /nix/macos/overview
[nix-daemon]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-daemon
[nix-enable-opt]: https://daiderd.com/nix-darwin/manual/index.html#opt-nix.enable
[nix-darwin]: https://github.com/LnL7/nix-darwin
[pr-1313]: https://github.com/LnL7/nix-darwin/pull/1313
[pr-1326]: https://github.com/LnL7/nix-darwin/pull/1326
[survival-mode]: /blog/nix-survival-mode-on-macos
---
**Changelog: improved nix-darwin support for Determinate!**
Published: February 18, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-033
We're excited to announce a handful of quality-of-life improvements to [Determinate]:
* Determinate now ships with [Nix 2.26.2][2.26.2].
* [FlakeHub Cache][cache] is no longer added to your [`nix.conf`][nix.conf] substituters until you're logged in, eliminating the confusing `not a binary cache` error.
* Much improved [nix-darwin] support!
People who use both [Determinate] and nix-darwin should now set `nix.enable = false` in their nix-darwin configuration.
This enables Determinate to handle Nix installation and configuration rather than nix-darwin, as was the case before.
Many gracious thanks to [Emily] on the nix-darwin team for helping us with this.
We'll have much more to say about this soon.
This release also has two bug fixes specifically targeted to enterprise users:
* Enterprise [macOS] users with [Time Machine][time-machine] disabled by [Mobile Device Management][mdm] (MDM) can now install Determinate.
* Users installing Nix from the Determinate macOS package are now notified in more cases if MDM prevents mounting volumes.
And finally, we also shipped a number of reliability improvements for the Determinate [installer]:
* We can now tolerate and mount the Nix Store APFS volume if it's unlocked but not yet mounted.
* Uninstalling Determinate now cleans up the generated `nix.conf` file.
* Invalid or corrupted `nix.conf` files no longer block installation or upgrades.
* Creating `/etc/profiles.d/nix.sh` now creates `/etc/profiles.d` if it doesn't exist already.
* The installation process is now far more resilient to strange behavior on XFS while unpacking the Nix tarball.
To [upgrade] Determinate now:
```shell title="Upgrade Determinate Nix"
sudo determinate-nixd upgrade
```
If you aren't using Determinate yet, check out our getting started guides for [organizations] and [individuals].
[2.26.2]: https://discourse.nixos.org/t/nix-2-26-released/59211
[cache]: https://flakehub.com/cache
[determinate]: https://docs.determinate.systems
[emily]: https://github.com/emilazy
[individuals]: https://docs.determinate.systems/getting-started/individuals
[installer]: https://github.com/DeterminateSystems/nix-installer
[macos]: /blog/tags/macos
[mdm]: https://docs.determinate.systems/guides/mdm
[nix.conf]: https://flakehub.com/nix-conf
[nix-darwin]: https://github.com/LnL7/nix-darwin
[organizations]: https://docs.determinate.systems/getting-started/organizations
[time-machine]: https://support.apple.com/104984
[upgrade]: https://docs.determinate.systems/determinate-nix#determinate-nixd-upgrade
---
**Improved evaluation times with pre-resolved Nix store paths**
Published: February 12, 2025
URL: https://determinate.systems/blog/resolved-store-paths
Building things with Nix is pretty amazing.
You point Nix at a flake reference for a derivation and say "build!"
It then spins up a sandbox and [reproducibly][reproducibility] creates real bytes on disk.
Those bytes on disk could be a single `hello.txt` file or a hefty Docker image or a fully functional [NixOS] system—or much else besides.
But the power of `nix build` comes at a cost, and not just in terms of disk space.
Before it writes anything to disk, Nix first needs to evaluate the [store path][paths] of the derivation that it's going to [realise][realisation].
It then uses that store path to determine whether it can pull the path from a [cache][caching] like [FlakeHub Cache][flakehub-cache] or whether it needs to build it instead.
Finally, it applies this logic recursively throughout the entire dependency tree of the derivation and you end up with the desired bytes on disk.
## The evaluation tax
Even if you can (ideally) pull a path from a cache instead of building it, you still have to pay Nix's *evaluation tax* in determining the derivation's store path in the first place.
This cost is no big deal if you're on a modern workstation or a beefy build server.
But on resource-constrained devices, such as smart sensors, Raspberry Pis, or industrial controllers, even pulling from a cache can use a ton of memory if you need to evaluate the store path—something you'd surely like to avoid at all costs.
## The solution: resolved store paths \{#solution}
Fortunately, [FlakeHub] now offers an elegant solution to this problem: [**resolved store paths**][docs].
With this feature, you can do two things that have not yet been possible in the Nix ecosystem:
1. You can provide FlakeHub with a [flake output reference][flakeref] and get the store path for that reference *without using Nix*.
With the store path in hand, you can pull directly from [FlakeHub Cache][flakehub-cache] *with no evaluation tax*.
1. You can apply a [NixOS], [Home Manager][hm], or [nix-darwin] configuration to the current host solely with a flake reference—again pulled directly from FlakeHub Cache without incurring an evaluation tax.
Resolving flake outputs is useful only in conjunction with [FlakeHub Cache][flakehub-cache], which is available if you [sign up][signup] for a [FlakeHub] paid plan.
Paid plans also provide access to features like [private flakes][private-flakes].
## Working with resolved store paths
You can work with resolved store paths using [fh], the CLI for FlakeHub, which enables you to [fetch store paths](#fh-resolve) and [apply configurations from flake references](#fh-apply).
## `fh resolve` \{#fh-resolve}
[fh], the CLI for FlakeHub, has a [`resolve`][fh-resolve] command that resolves [flake references][flakeref] to store paths by fetching them directly from FlakeHub.
Here's an example:
```shell title="Resolving the path to a NixOS configuration"
fh resolve \
"DeterminateSystems/store-paths/*#nixosConfigurations.baseline"
```
It also works with the full flake reference:
```shell title="Resolving the path to a NixOS configuration with a full flake reference"
fh resolve \
"https://flakehub.com/f/DeterminateSystems/store-paths/*#nixosConfigurations.baseline"
```
Both of these fh commands return this store path immediately:
```
/nix/store/84g3x33v5srlljhfaskwy7mmar5697vn-nixos-system-minimal-24.05.20240714.53e81e7
```
It may not be clear why this is special, as you *can* get the same result using plain old [`nix eval`][nix-eval]:
```shell title="Get store path info"
nix eval --raw "https://flakehub.com/f/DeterminateSystems/garage-door/0.1.9#nixosConfigurations.turner.config.system.build.toplevel"
```
But this command requires Nix to be available and takes several seconds and a fair amount of memory and disk space to complete.
With `fh resolve`, by contrast, fh fetches an *already calculated store path* stored in FlakeHub's database.
In a variety of deployment scenarios, using fh is a vastly superior option.
## `fh apply` \{#fh-apply}
[fh], the CLI for FlakeHub, has an [`apply`][fh-apply] command that you can use to apply [NixOS](#nixos), [Home Manager](#home-manager), and [nix-darwin](#nix-darwin) configurations to the current system.
`fh apply` is powered by the same path resolution feature mentioned above, meaning that you can use it to, for example, apply a [NixOS](#nixos) configuration to the current host.
### NixOS
Let's say that you have a [NixOS] configuration as a [flake output][output] on your `my-org/nixos-configs` repo, while that configuration is being pushed to [FlakeHub Cache][flakehub-cache].
With [fh], you could apply that configuration using this single command:
```shell title="Apply a NixOS configuration from a flake output reference"
fh apply nixos "my-org/nixos-configs/0.1#nixosConfigurations.staging-server"
```
If you leave off the flake output reference—`nixosConfigurations.staging-server` in this case—fh "magically" infers that you want the `nixosConfigurations.$(hostname)` output.
If that's the case, you can shorten the reference to this:
```shell title="Apply a NixOS configuration from a flake output reference"
fh apply nixos "my-org/nixos-configs/0.1"
```
### NixOS on Amazon Web Services (AWS) \{#nixos-aws}
The value of this new approach becomes clear in this little snippet of code:
```shell title="Log in to AWS and apply a NixOS configuration"
determinate-nixd login aws
fh apply nixos "my-org/nixos-configs/0.1"
```
If you have an [EC2] instance with [Determinate Nix][det-nix] installed, this is all you need to run to update your NixOS system configuration.
[Determinate Nixd][dnixd] logs you in, which enables you to use [private flakes][private-flakes], and then `fh` does its thing.
As before, it does so without needing to evaluate the store path, which makes this speedier than your typical NixOS deployment.
And because FlakeHub supports [semantic versioning][semver] (SemVer) for flakes, the `fh apply` operation would automatically use the most recent release within the supplied version constraint, so you could also use constraints like `*` (whatever is latest), `=0.1.2` (exact match), or `~1.2` (flexible patch).
### Home Manager
While `fh apply` for NixOS works swimmingly in cloud environments like AWS, it also supports applying [Home Manager][hm] configurations.
Here's an example:
```shell title="Apply a Home Manager configuration from a flake output reference"
fh apply home-manager "my-org/home-configs/0.1#homeConfigurations.justme"
```
`fh apply` has a default flake output name of `homeConfigurations.$(whoami)`, so this command without the flake output would work if the output name conformed to that:
```shell title="Apply a NixOS configuration from a simplified reference"
fh apply home-manager "my-org/home-configs/0.1"
```
### nix-darwin
And finally, `fh apply` also supports [nix-darwin].
We at [Determinate Systems][detsys] are committed to providing first-class Nix support on macOS and this is a testament to that.
Here's an example command:
```shell title="Apply a nix-darwin configuration from a flake output reference"
fh apply nix-darwin "my-org/macos-configs/0.1#darwinConfigurations.justme-aarch64-darwin.system"
```
The default flake output name for nix-darwin is `darwinConfigurations.${hostname}`, where `hostname` is the value of `scutil --get LocalHostName`.
If your flake output name conforms to that, you can run this:
```shell title="Apply a nix-darwin configuration from a simplified reference"
fh apply nix-darwin "my-org/macos-configs/0.1"
```
## Generating store paths
Above, we saw how to use [`fh resolve`](#fh-resolve) and [`fh apply`](#fh-apply) with resolved store paths.
To be able to fetch those paths, however, you need to generate them as part of the [flake publishing process][publishing].
You can configure that with a one-liner in the [flakehub-push] Action:
```yaml title=".github/workflows/flakehub.yml" {4}
- uses: DeterminateSystems/flakehub-push@main
with:
name: my-org/my-flake
include-output-paths: true
```
With this configuration in place, the store paths for all of your flake's outputs are evaluated and stored in FlakeHub's database.
As with everything on FlakeHub, those paths are available only to users with proper permissions.
For [public flakes][public-flakes], that's anyone with [fh] installed; for [private flakes][private-flakes] that's only those to whom you've explicitly granted access.
Here's a full GitHub Actions workflow with resolved store paths enabled:
```yaml title=".github/workflows/flakehub.yml" {23}
name: Publish flake with resolved store paths available
on:
push:
branches:
- main
jobs:
flakehub-publish:
runs-on: ubuntu-22.04
needs: build-docker-image
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/flakehub-push@main
with:
name: omnicorp/web
rolling: true # Publish on every push to the default branch
visibility: private
include-output-paths: true
```
With that output path published, users with access to the `omnicorp/web` flake could use fh to fetch a resolved path like this:
```shell title="Resolve store path after publishing"
fh resolve "omnicorp/web/*#dockerImages.x86_64-linux.server"
```
## Truly new horizons for Nix
[FlakeHub] now enables you to determine store paths without evaluating them, which unlocks some exciting new possibilities for Nix in the enterprise.
With this feature, [`fh apply`](#fh-apply) can apply Nix-based configurations for [NixOS](#nixos), [Home Manager](#home-manager), and [nix-darwin](#nix-darwin) in a much more efficient way than is currently available in the Nix ecosystem.
Beyond `fh apply`, we can imagine people using [`fh resolve`](#fh-resolve) for things like scripting or pulling artifacts like Docker containers from [FlakeHub Cache][cache] (more on this soon).
[cache]: https://flakehub.com/cache
[caching]: https://zero-to-nix.com/concepts/caching
[det-nix]: https://docs.determinate.systems/determinate-nix
[detsys]: https://determinate.systems
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[docs]: https://docs.determinate.systems/flakehub/store-paths
[ec2]: https://aws.amazon.com/ec2
[fh]: https://docs.determinate.systems/flakehub/cli
[fh-apply]: https://docs.determinate.systems/flakehub/cli#apply
[fh-resolve]: https://docs.determinate.systems/flakehub/cli#resolve
[flakehub]: https://flakehub.com
[flakehub-cache]: https://docs.determinate.systems/flakehub/cache
[flakehub-push]: https://github.com/DeterminateSystems/flakehub-push
[flakeref]: https://zero-to-nix.com/concepts/flakes/#references
[hm]: https://flakehub.com/flake/nix-community/home-manager
[nix-darwin]: https://github.com/LnL7/nix-darwin
[nix-eval]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-eval
[nixos]: https://zero-to-nix.com/concepts/nixos
[output]: https://zero-to-nix.com/concepts/flakes#outputs
[paths]: https://zero-to-nix.com/concepts/nix-store/#store-paths
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[public-flakes]: https://docs.determinate.systems/flakehub/concepts/visibility#public
[publishing]: https://docs.determinate.systems/flakehub/publishing
[realisation]: https://zero-to-nix.com/concepts/realisation
[reproducibility]: https://zero-to-nix.com/concepts/reproducibility
[semver]: https://docs.determinate.systems/flakehub/concepts/semver
[signup]: https://flakehub.com/signup
---
**Supercharging Nix deployments with FlakeHub Cache**
Published: January 24, 2025
URL: https://determinate.systems/blog/home-manager-deployments-with-fh
Today, I'm excited to share how [FlakeHub Cache][cache] can dramatically improve your Nix development *and* deployment workflow.
While our [Magic Nix Cache][mnc] has helped teams boost CI performance over the past year, FlakeHub Cache represents the next evolution - accelerating builds and fundamentally transforming how Nix deployments work.
We'll examine a [Home Manager][hm] configuration that demonstrates how FlakeHub Cache's pre-built [closures] and pre-evaluated store paths can make deployments both faster and more reliable, especially in resource-constrained environments.
If you're currently using Magic Nix Cache, which will [reach end-of-life for its free tier][mnc-eol] on **February 1st, 2025**, this is the perfect time to explore how FlakeHub Cache can enhance your entire Nix workflow.
## The challenge with traditional Nix deployments
If you've worked with Nix, you're likely familiar with the standard deployment flow: evaluate Nix expressions, [realise][realisation] closures, and download or build missing dependencies.
While this works, it can be time-consuming and resource-intensive, especially when deploying to multiple machines or CI environments.
## Enter FlakeHub Cache
FlakeHub Cache fundamentally changes how Nix deployments work by eliminating build time and evaluation overhead.
Here's why this matters:
Traditional Nix deployments, even with a binary cache, require two steps that can be resource-intensive:
- **Evaluation**: Nix must evaluate your configuration to determine the store paths it needs - this alone can overwhelm memory-constrained devices
- **Realization**: Nix then either builds those paths or downloads them from a binary cache
FlakeHub Cache eliminates both steps.
When you push configurations through CI, FlakeHub:
- Evaluates and stores the exact store paths your configuration needs
- Caches the built closures for those paths
Using FlakeHub Cache when deploying means you can bypass Nix evaluation entirely:
```shell title="Apply the Home Manager configuration supplying only a flake output"
fh apply home-manager "determinatesystems/home-manager-example/*#homeConfigurations.linuxUsername@linuxHostname"
```
This single command is transformative - especially for resource-constrained environments where evaluating a Nix configuration could cause out-of-memory errors.
FlakeHub already knows the store paths needed, so no local Nix evaluation is required.
It fetches the pre-built closure and activates it.
## Real benefits for real teams
What makes this approach particularly powerful?
- **Zero evaluation overhead**: Skip not just building but even the memory-intensive process of evaluating Nix expressions
- **Resource efficient**: Deploy to severely constrained environments that couldn't even evaluate complex Nix configurations
- **Consistent environments**: Every machine gets bit-for-bit identical outputs
- **CI verified**: Configurations are pre-tested before reaching production
- **Bandwidth optimized**: Only download what you need
## See it in action
We've created a complete example repository that demonstrates these concepts.
The configuration supports both macOS and Ubuntu systems, showing how to:
- Structure your flake for multi-platform support
- Configure GitHub Actions to build and cache your configurations
- Deploy configurations rapidly using `fh`
Check out the [complete example on GitHub][hm-example] to see how it all fits together.
## Beyond Home Manager
While this example focuses on Home Manager, the same principles apply to:
- NixOS system configurations
- nix-darwin configurations
- Development environments
- Custom tooling and packages
The benefits become even more pronounced with larger configurations that traditionally require significant build time.
## Getting started
Ready to supercharge your own Nix deployments? Here's how to get started:
1. [Sign up for FlakeHub][signup] to get access to FlakeHub Cache.
2. Get one month free by applying the coupon code **FHC** during checkout. If you're a maintainer of an open source project, contact us at `support@flakehub.com` for a free account.
3. Review our [comprehensive documentation][flakehub-docs] to learn more about FlakeHub's capabilities.
4. Follow our [getting started guide][getting-started] to begin your journey.
## Join the revolution
FlakeHub Cache represents a fundamental shift in how we think about Nix deployments.
By moving computation from deployment time to build time, we're making Nix more accessible and practical for teams of all sizes.
Have questions or want to learn more? Join us on [Discord].
We'd love to hear about your use cases and help you get started with FlakeHub Cache.
Remember: your deployments should be boring and predictable, providing a sense of security and stability.
With FlakeHub Cache, they can be.
[cache]: https://docs.determinate.systems/flakehub/cache
[closures]: https://zero-to-nix.com/concepts/closures
[discord]: https://determinate.systems/discord
[flakehub-docs]: https://docs.determinate.systems/flakehub
[getting-started]: https://docs.determinate.systems/getting-started
[hm]: https://github.com/nix-community/home-manager
[hm-example]: https://github.com/DeterminateSystems/home-manager-example
[realisation]: https://zero-to-nix.com/concepts/realisation
[signup]: https://flakehub.com/signup
[mnc]: https://github.com/DeterminateSystems/magic-nix-cache
[mnc-eol]: /blog/magic-nix-cache-free-tier-eol
---
**End of life for the free tier of the Magic Nix Cache**
Published: January 21, 2025
URL: https://determinate.systems/blog/magic-nix-cache-free-tier-eol
We recently became aware that the API backing [Magic Nix Cache][mnc]'s free CI cache in [GitHub Actions][actions] will be shut down on February 1st, 2025.
This will require most users to change their [GitHub Actions][actions] workflows.
The TL;DR:
* The free Magic Nix Cache will stop working unless you're on [GitHub Enterprise Server][gh-enterprise].
* Users can upgrade to [FlakeHub Cache][flakehub-cache] and get **one month free** using the coupon code `FHC`.
* Open source projects can request a free account at [support@flakehub.com][email].
## Background
When we set out to build Magic Nix Cache, we studied the code of the [GitHub Actions Cache][gha-cache] and found that the API for the backend was relatively trivial, consisting of just a few standard HTTP calls.
With the help of [Attic]'s creator, [Zhaofeng Li][zhaofeng], we created Magic Nix Cache based on that API.
Last September, GitHub published a [blog post][gha-post] about shutting down that API.
On December 5th, 2024, GitHub merged and released an updated Cache Action using the [new API][new-api-pr].
Unfortunately, we didn't see either blog post or the updated API until last week, which gives us limited time to respond.
## Analysis
Some information we learned when researching this new API:
* GitHub's new cache API is built on [Twirp], a [Protocol-Buffers][protobuf]-based RPC framework built by [Twitch].
* Their updated client code became available first in September and was merged in late November.
* The new client code is generated from a Protocol Buffers `.proto` file, but the original source `.proto` file has [not been made available][proto-file-issue].
Before we released Magic Nix Cache, we contacted GitHub in every way we could to see if it was acceptable.
We never heard back, assumed that no news is good news, and moved forward.
Had GitHub released their `.proto` files, we would have a clear path forward—but they didn't.
Whether or not GitHub is even *aware* of Magic Nix Cache, I believe that GitHub is sending a clear message that they don't want tools like Magic Nix Cache to integrate with their cache the way that we have.
The only options we have at this point are:
* Rewrite Magic Nix Cache from scratch in [TypeScript] and use the [Actions Toolkit][gh-sdk].
* Reverse engineer the `.proto` source to generate a [Rust] client.
* Discontinue Magic Nix Cache's free GitHub-Actions-based cache.
One of the risks of repurposing an API in an unexpected way is that it can change and break on short notice.
And because it *is* such short notice, I don't see the first two options working out in the timeline available to us.
## The future
I hope that we can bring back Magic Nix Cache's free cache for GitHub Actions.
It was the most frictionless way to get a free binary cache to make Nix CI faster.
This is not what I hoped for, and I would be happy to see GitHub release their `.proto` files, which would enable us to update Magic Nix Cache to use it.
In the future, we may also reimplement it in TypeScript using their official [Actions Toolkit][gh-sdk].
## The present
We are not able to migrate Magic Nix Cache to use their new APIs in the timeline available.
Because of that, users of the free version of Magic Nix Cache will start seeing their CI runs fail as soon as GitHub sunsets their v1 cache API.
Any builds still using the [Magic Nix Cache Action][mnca] will begin to **fail** once GitHub proceeds with decommissioning the v1 cache API.
We have opted to fail builds rather than allowing them to proceed quietly without using the cache because we want to prevent those builds from silently using costly build minutes.
[GitHub Enterprise Server][gh-enterprise] users can continue to use Magic Nix Cache as before, as GitHub Enterprise Server still has the old v1 cache API available, for now.
If you'd like to continue using a Nix [binary cache][cache] in GitHub Actions, you can switch to [FlakeHub Cache][flakehub-cache] and get one month free by applying the coupon code `FHC` during checkout.
If you're a maintainer of an open source project, contact us at [support@flakehub.com][email] for a free account.
[actions]: https://docs.github.com/actions
[attic]: https://github.com/zhaofengli/attic
[cache]: https://zero-to-nix.com/concepts/caching
[email]: mailto:support@flakehub.com
[flakehub-cache]: https://flakehub.com/cache
[gh-enterprise]: https://github.com/enterprise
[gha-cache]: https://github.com/actions/cache
[gha-post]: https://github.blog/changelog/2024-09-16-notice-of-upcoming-deprecations-and-changes-in-github-actions-services
[gh-sdk]: https://github.com/actions/toolkit
[mnc]: https://github.com/determinatesystems/magic-nix-cache
[mnca]: https://github.com/determinatesystems/magic-nix-cache-action
[new-api-pr]: https://github.com/actions/cache/pull/1509
[proto-file-issue]: https://github.com/actions/toolkit/issues/1931
[protobuf]: https://protobuf.dev
[rust]: https://rust-lang.org
[twirp]: https://github.com/twitchtv/twirp
[twitch]: https://twitch.tv
[typescript]: https://typescriptlang.org
[zhaofeng]: https://github.com/zhaofengli
---
**Changelog: customer binding, nix.conf, and yanked Nix 2.25.4**
Published: January 14, 2025
URL: https://determinate.systems/blog/changelog-determinate-nix-030
We'd like to announce some noteworthy changes to [Determinate Nix][det-nix] involving [customer binding](#binding), [user customization](#customization), [Determinate in OCI containers](#containers), and a [yanked Nix release](#yanked-nix-release).
We've also cut two new releases for the [Determinate Nix Installer](#installer-releases).
### Customer binding \{#binding}
We've updated [Determinate Nixd][dnixd] to enable users to bind a [Determinate] installation to a specific [FlakeHub] organization.
When you do this, it prevents users from logging in to a FlakeHub account that isn't associated with the organization.
Here's an example binding command:
```shell title="Binding a Determinate installation to a specific org"
determinate-nixd auth bind AmbiguousTechnologies
```
For persistent CI workers, we recommend pre-binding the [Determinate] installation to prevent CI jobs from substituting from outside caches.
For more information, see our documentation on [binding your installation][binding].
### Splitting user customization from Determinate defaults in the Determinate Nix Installer \{#customization}
The [Determinate Nix Installer][installer] now writes user-specified configuration entries to a `nix.custom.conf` file.
The default Nix configuration is written to `nix.conf`, which imports the `nix.custom.conf`.
We made this change in [collaboration with Emily from the nix-darwin project][nix-darwin-collab], for which we are grateful.
At the same time, Determinate now takes a more prescriptive approach to the `nix.conf`.
Previously, Determinate left the `nix.conf` alone if it was a symlink.
This created unpredictable behavior for some users.
Determinate will now move the `nix.conf` out of the way to a backup file and write its own at startup.
User-specified settings can still be saved to `nix.custom.conf`.
Determinate also previously tried to migrate the contents of `nix.conf` to the custom file, but the migration was unacceptably sloppy and error prone.
Users reported seeing their `nix.custom.conf` growing with time, which is certainly not ideal.
Consider reviewing your custom configuration and deleting any duplicate entries.
### Determinate in OCI containers \{#containers}
While installing upstream Nix with the [Determinate Nix Installer][installer] generally works quite well in [Open Container Initiative][oci] (OCI) containers, installing [Determinate] has generally not worked properly and we are not currently testing for container compatibility.
If your use case involves Determinate in OCI containers, [reach out][email] to us and we can put improved container support on our roadmap.
### Nix 2.25.4 yanked from DeterminateSystems/nix on FlakeHub \{#yanked-nix-release}
During our release process for Nix 2.25.4, we saw a [regression] in Nix around relative `path:.` flake references.
Nix 2.25.4 was briefly made available through our [`nix`][nix] and [`determinate`][determinate] flakes.
Those releases have been [yanked][yanked-releases] and rolled back to [2.25.3][non-yanked-version].
We [submitted a patch][patch] for the bug and will validate and roll out Nix 2.25.5 as soon as possible.
## Determinate Nix Installer releases \{#installer-releases}
In addition to changes to Determinate Nix, we recommend checking out these recent releases of the Determinate Nix Installer:
* [v0.33.0](https://github.com/DeterminateSystems/nix-installer/releases/tag/v0.33.0)
* [v0.32.3](https://github.com/DeterminateSystems/nix-installer/releases/tag/v0.32.3)
[binding]: https://docs.determinate.systems/determinate-nix#determinate-nixd-binding
[determinate]: https://flakehub.com/flake/DeterminateSystems/determinate
[det-nix]: https://docs.determinate.systems/determinate-nix
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[email]: mailto:hello@determinate.systems
[flakehub]: https://flakehub.com
[installer]: https://github.com/DeterminateSystems/nix-installer
[nix]: https://flakehub.com/flake/DeterminateSystems/nix
[nix-darwin-collab]: https://github.com/LnL7/nix-darwin/pull/1266
[non-yanked-version]: https://flakehub.com/flake/DeterminateSystems/nix/2.25.3
[oci]: https://opencontainers.org
[patch]: https://github.com/NixOS/nix/pull/12254
[regression]: https://github.com/NixOS/nix/issues/12248
[yanked-releases]: https://docs.determinate.systems/flakehub/concepts/yanked-releases
---
**Determinate Nix now supports migrating from other caches**
Published: December 18, 2024
URL: https://determinate.systems/blog/flakehub-cache-migration
In October, we [released Determinate Nix][announcing-determinate-nix] and announced that [private flakes and FlakeHub Cache][ga-flakehub] are now generally available.
FlakeHub authentication is often automatic, using cloud native mechanisms like [JSON Web Tokens][jwt] (JWTs) and [Amazon Instance Metadata Service][imds] (IMDS), eliminating from your workflows cumbersome tasks like sharing credentials and manually rotating tokens.
As a stark alternative, Determinate Nix manages the lifecycle of these tokens by automatically synthesizing a [netrc] file that Nix can use to authenticate with FlakeHub.
That means that your entire organization has a frictionless path to using features like [FlakeHub Cache][cache] and [private flakes][private-flakes].
Since that initial release, customers migrating to FlakeHub Cache have indicated that they want to read from their old cache during the transition.
These customers are excited by FlakeHub Cache's support for [Single sign-on][sso] (SSO) and rotating per-user credentials but they still have static credentials to their fleet of machines shared amongst their teams.
To serve these customers—any another potential FlakeHub users—we now support extending our synthesized netrc file with customer-defined entries, and we recently published a new guide for [Migrating to FlakeHub Cache][migrating].
Sign up now at https://flakehub.com/signup and visit our [getting started docs][getting-started] to get the best Nix workflow for your team.
And as always, we'd love to hear your feedback on our Discord at https://determinate.systems/discord.
[announcing-determinate-nix]: /blog/announcing-determinate-nix
[cache]: https://docs.determinate.systems/flakehub/cache
[ga-flakehub]: /blog/flakehub-cache-and-private-flakes/
[getting-started]: https://docs.determinate.systems/getting-started
[imds]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html
[jwt]: https://jwt.io
[migrating]: https://docs.determinate.systems/guides/migrate-to-flakehub-cache
[netrc]: https://everything.curl.dev/usingcurl/netrc.html
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[sso]: https://en.wikipedia.org/wiki/Single_sign-on
---
**The future of software is Nix**
Published: October 25, 2024
URL: https://determinate.systems/blog/the-future-is-nix
The future of software is [Nix][z2n-nix] and we at Determinate Systems want to have a role in building that future.
Today, I want to talk about [Determinate], and I want to speak to you from a more personal place than you might be accustomed to here on our blog.
## Determinate, to live up to the promise of our industry
What if every machine had a modern Nix with [flakes enabled][nix-installer] out of the box?
And the [cache] just worked?
And [private repositories][private-flakes] were straightforward to use?
And Nix was up to date?
And updating your dependencies took little to no effort?
And the Nix store didn't occasionally take up your entire disk, with no clear path to solving that?
What if adopting Nix were straightforward?
What if adopting Nix didn't involve solving a zillion annoying puzzles?
What if adopting Nix didn't irritate your IT or security teams?
That's our vision.
To make a smooth path from "Nix could solve my dev, build, and configuration management problems" to "Nix _is_ solving these problems" and in a way that doesn't involve turning someone from a developer to "the Nix person."
For users to pick up Nix and be effective and deploy critical software and not have to spend every waking moment solving unrelated puzzles.
For users to have components that fit together in straightforward ways to let them actually do the thing that is important—where the important thing is *not* toying with Nix.
A future where software is maintainable and secure and we can take big leaps and still be safe and confident it'll be fine.
Where we can take risks and undo it.
Where folks building critical infrastructure or deploying to space feel confident choosing Nix.
Where Nix is mature enough for their use case.
Where security teams know where to find clearly communicated, timely, advisories?
That's why we're here.
To make more Nix users.
To make those users happier and more productive.
To make critical infrastructure safer.
## How I got here
I knew Nix was the future in 2016 when I found Nix and switched all my personal systems and work infrastructure to it within just a few weeks.
And I know this now more than ever.
I can feel it in my bones.
## Immutable infrastructure is the way
By 2014 I had been burned by declarative configuration management tools like [Chef] and [Puppet].
That experience spurred me to convince the startup where I worked to aggressively adopt immutable infrastructure.
With Chef, I had experienced severe drift not just from unmanaged config changes but also because the world around us was changing.
I'd come in the next day, push out a change with Chef, and the unmanaged configuration met the new managed configuration and kaboom: production outage.
I'd also been burned by the [Docker]'s promise of reproducibility.
The company where I worked at the time was in the process of aggressively adopting Docker for all of its deployments.
Every merge to one of our dozens of repositories created fresh Docker images.
We pushed those images to a registry and then kicked off a CI/CD pipeline that would [use Packer to build a brand new AMI][packer-ami].
We'd then do a blue/green-style rolling deployment to our autoscaling groups to finalize the deploy.
It was brilliant.
I started talking at conferences about the importance of immutable infrastructure and about how incredibly stable and resilient our deployments and systems were.
### Chef and Docker weren't enough for great immutable infrastructure
But our CI/CD pipeline was slow.
And it broke—a lot.
We learned two things:
1. Trying to manage individual packages or general system state with [Chef] was really hard.
An annoying but solvable part of this came from resources like users where, if you added a user then wanted to remove them, you had to explicitly write that "remove" code.
The bigger issue was that steps like "install nginx" could do different things on different servers that were running applying updates at the same time.
I'd seen that several times, where we'd be deploying changes at the same time the apt mirrors were updating.
2. `Dockerfile`s weren't reproducible because the entire world around you was changing.
Files would disappear or change from URLs.
Base images would change randomly.
It was common practice to run `apt-get update` in your image, and who knows what that was doing?
At conferences, I started talking about how Docker images were not enough because of how much the world was changing under our feet.
How Chef and Puppet and similar tools weren't enough because of how much of the system was unmanaged.
How the root of this problem is how blind we were as operators were to the details of our software and systems.
I remember paraphrasing Carl Sagan a lot:
> If you want to reproducibly build a server from scratch, you first have to pin the universe.
### I've seen these words before...
In January 2016 I was complaining to a close friend about how slow it was to build a new AMI with Chef.
She pointed me to the [NixOS website][old-nixos-org].
I read it—and I was Not Impressed:tm:.
Beyond the boring website, it said words I'd seen before.
Those words I'd read about Chef and Docker. "Declarative." "Reliable." "Roll back."
Yeah, yeah.
Those words were heavily in vogue at the time.
I'd heard them before, and they were lies.
But my friend persisted.
"I don't know, Graham... You should probably try it..."
### I tried it
By the time I found [NixOS], I knew what a system needed to look like if it were to be reproducible, declarative, effective, and to actually accomplish these goals we were trying to achieve as an industry:
1. Strictly enforcing that all inputs are hashed.
1. Eliminating network access during the build.
1. Enabling you to build the system you want instead of trying to change a running system.
And here it was.
Nix made sense.
Nix was fast.
Nix was declarative.
Nix was reliable.
Nix's rollbacks really did work.
I "got it."
I was hooked.
This was the future.
I had to *make this* the future.
With all love and respect to the Chef and Docker ecosystems, which taught me so much and to this day extend great generosity: I never wanted to use either tool again.
## All in
I tried NixOS on a Wednesday and by Saturday I had erased macOS and switched to NixOS.
{/* vale off */}
It wasn't easy!
{/* vale on */}
Within a few weeks, I'd replaced thousands of lines of Chef cookbooks with a few hundred lines of Nix expressions and a Bash script to build and upload AMIs to AWS.
The next step for me was fixing the obvious problem: Nix was hard to adopt.
### Packaging was hard
A major problem was packaging software in the first place.
So much of the software ecosystem around Nix hadn't even started [pinning] dependencies or writing out version locks.
In a sense, the world was starting to realize what Eelco had identified over ten years prior: that pinning versions and hashing dependencies was not just worth doing, but critical to software security.
This has been a big focus of the industry's work in the last fifteen plus years, but at the time there was so much to do.
We're in a different place than we were at the time.
{/* vale off */}
The industry has moved "towards" Nix in so many ways, cutting the packaging difficulty from "impossible" to "easy" for so many languages.
{/* vale on */}
### Documentation was hard
The docs were written largely by academics or folks so deeply steeped in Nix their writing was impenetrable to outsiders.
This has improved since but there is still plenty to be done.
This problem was not just practical "how-to" documentation.
Nix didn't have great documentation to communicate and teach why Nix was important and why its rules were important.
It struggled to educate the world about the new way of working that it demanded and about the incredible gifts it had bestowed on its users.
I would not have fallen in love with it if I had not already made the conceptual leaps I'd already made.
### Persisting was so rewarding
Everything about Nix was a hugely rewarding puzzle.
Every successful build was a huge dopamine hit, rewarding the challenges of navigating the Nix libraries and fixing build problems.
Every pull request to [Nixpkgs] was quickly reviewed, merged, and *celebrated* by a relatively small group of dedicated folks who loved this tech as much as I did.
## Even more "in"
After a couple years of intensely contributing to the project in my spare time, I quit my job to do full-time Nix consulting.
Consulting taught me some important things.
### Consulting revealed that Nix's flexibility made it too hard
As a Nix consultant, I would often drop into a project and team that had more or less successfully and effectively adopted Nix.
Invariably, that team used Nix in a way that was wholly their own and completely unlike that of any other team I'd worked with.
Their Nix champion had followed a choose-your-own-adventure game of writing scripts and tools and `.nix` files to adopt Nix at their company.
But this was not their fault.
There were no standard processes, no workflows, no off-the-shelf techniques to adopt and be successful.
On top of that, a lot of Nixpkgs' interfaces were undocumented and hard to discover.
### Consulting revealed that Nix just wasn't quite mature enough
Nix's flexibility meant joining a new team meant learning their own unique approach.
As I worked with more and more serious use cases, I noticed a schism in expectations and reality.
These problems compounded as I moved up into increasingly sensitive and critical use cases.
They loved what Nix got them but they would squirm when we talked about how pull requests were merged.
How casual the review process was.
How casual the "maintainer" role was treated.
How casually security patching was handled.
It was uncomfortable to beat around the bush and say "well, historically it has been fine ..." and "folks are usually careful about updates..."
And for security patching, corporate users had security and compliance objectives that didn't run on vibes alone.
They needed to know what was vulnerable and what wasn't.
They needed to know when security updates were available and how to get them.
## Let's get mature
I became obsessed with maturing the project.
I wrote [OfBorg] to catch more contributor errors before they merged.
I started the NixOS security team and ran weekly security round-ups until our data source literally shut down.
I wrote security advisories.
I rewrote the Nix installer to be more defensive and reliable.
I worked like crazy to make the Nix installer use the daemon for better security, and to be more defensive and reliable.
The bones of that installer are still the foundation of today's upstream Nix installer.
To be clear, while I did start and lead these (and other) initiatives and my fingerprints of effort and love are all over the project, I didn't do any of this alone.
Thank you, to the entire community, for which all of this would be impossible.
## The world *needs* Nix
I remember the moment I realized I was white-knuckling the project into maturing.
Something in Nixpkgs broke.
It wasn't a particularly big deal but it did need to be treated seriously and fixed.
Someone made a comment like "good thing this isn't being used by banks or whatever!"
At that moment, I was working with financial firms using Nix in their banking and trading platforms.
I was dizzied by the disparity between how important the project was compared to the concrete reality of the project.
The next 100 years of computing needs to start with Nix.
Our world's infrastructure should not be stuck in a "don't touch it, it works" mode, but it is.
And I don't just mean low-stakes systems.
Our entire world is software, and as an industry we're failing the world.
Critical infrastructure running power grids and nuclear reactors and robots and scientific research is woefully insecure because it is {/* vale off */}simply{/* vale on */} too hard and scary to maintain.
Nix can, and does, solve many of these problems in real terms today.
I felt at the time that it was a huge tragedy that Nix wasn't mature enough to be used in those scenarios.
I felt frustrated and occasionally angry that so many people saw the sparkling diamond in front of us but were afraid of reaching for the big picture.
This stuff matters.
### I love Nix. I hate that Nix doesn't have consistent interfaces.
Nix was so obvious and important.
And painful.
The inconsistencies really stacked up.
Not just across teams and projects and repositories, but across environments.
Every new target was a fresh batch of misery.
Setting up CI, or deploying to a new machine or cloud provider or bare metal server was a fresh batch of tiresome questions:
* How do I onboard my team members?
* How do I set up my cache in CI? Or on the new server?
* How do I deploy to this machine?
* How do I scale up and down quickly?
* How do I get private repositories in there?
* Why do I have to SSH in as root to deploy?
* How do I manage my NixOS infrastructure that feels like 2015 and not 2005?
It just went on and on and the answers invariably involved writing way too many Bash scripts predicated on hopes and dreams.
Maintenance was annoying and the trade-offs were deeply sub-optimal.
It is mind numbing how badly we were missing the table stakes of similar ecosystems.
I wanted a developer experience that didn't involve banging together hundreds of lines of copy-pasted Bash scripts to be good.
Nix's raison d'être is to make it possible to move software from one computer to another and have it keep working.
The promise was there, it worked, but the reality was occasionally misery.
#### Okay, fine, flakes
I didn't care for adopting [flakes], but only because they were experimental.
My thinking evolved.
I came around after realizing that flakes cut down on inconsistencies by making a standard evaluation model and input/output interface for projects.
Flakes are a first, small step, but a [foundationally important step][flakes-statement].
But a portion of the community has not made that leap yet.
I still think that even if flakes are occasionally annoying, or even if there are flaws in their design, they're predictable, improve build performance, add stability, and facilitate collaboration across developers and teams.
They make Nix viable, and delightful, at work.
There are blog posts out there that describe how to get flakes without flakes.
They're wrong because the interface is the point.
Everything else is details.
Today, we still find ourselves sitting on the most powerful technology of our lifetimes and we can't even decide on small steps in the direction of making it {/* vale off */}easier{/* vale on */} for folks to adopt it.
## Determinate, to live up to the promise of our industry
So here we are.
We're building it.
We're unabashedly making Nix the best way to build and deploy software to the most important targets in the world.
Our work means [flakes are enabled][nix-installer].
That a new user has a one-step process to get [private repositories][private-flakes] and access to their team's [cache].
We've made tools for tracking and updating dependencies.
Our documentation, tools, and workflow are designed top to bottom to make adopting Nix dramatically more straightforward.
We're working with IT and security teams to make Nix a welcome addition.
We're deliberately cutting out puzzles and bash scripts for deployment and access control in dev, test and prod.
We're building that smooth path from "Nix could solve these problems" to "Nix is solving these problems".
We're making it possible to use Nix for critical software, with components that fit together in straightforward ways.
We're helping teams focus on the valuable problems they're solving.
We're enabling teams to safely take big leaps with confidence and safety.
That's why we're here.
That's our vision.
To make amends and live up to the promise our industry has made to the world.
[cache]: /blog/flakehub-cache-and-private-flakes
[chef]: https://chef.io
[determinate]: https://docs.determinate.systems
[docker]: https://docker.com
[flakes]: https://zero-to-nix.com/concepts/flakes
[flakes-statement]: /blog/experimental-does-not-mean-unstable
[nix-installer]: https://github.com/DeterminateSystems/nix-installer
[nixos]: https://zero-to-nix.com/concepts/nixos
[nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs
[ofborg]: https://github.com/NixOS/ofborg
[old-nixos-org]: https://web.archive.org/web/20160115032501/https://nixos.org
[packer-ami]: https://grahamc.com/blog/packer-ami-device-volume-types
[pinning]: https://zero-to-nix.com/concepts/pinning
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[puppet]: https://puppet.com
[z2n-nix]: https://zero-to-nix.com/concepts/nix
---
**Nix at work: FlakeHub Cache and private flakes**
Published: October 23, 2024
URL: https://determinate.systems/blog/flakehub-cache-and-private-flakes
Last year, we here at Determinate Systems [announced FlakeHub][flakehub-announcement], a platform for publishing and discovering [Nix flakes][flakes].
From the outset, FlakeHub provided pathbreaking features like [semantic versioning][semver] (SemVer) for flakes and a variety of ways to [discover][discovery] flakes published to the platform.
In the meantime, numerous [Nix] users have published [hundreds of flakes][all-flakes] and used those flakes countless times—for their [Home Manager][hm] and [nix-darwin] configurations, to deploy [NixOS] systems to production, to distribute tools like [Tailscale] and [Colmena], and much more.
Today, we're delighted to announce a new chapter for FlakeHub with the general availability of two new features: [**FlakeHub Cache**](#flakehub-cache) and [**private flakes**](#private-flakes).
These powerful additions are now open to *all* FlakeHub organizations, helping you solve a variety of complex challenges that can arise when scaling [Nix] adoption across teams and projects.
## FlakeHub Cache
We [announced the private beta][beta] of FlakeHub Cache a while back and as of today anyone can [sign up][signup] and [get started][cache-docs].
Traditional [Nix binary caches][caching] present significant challenges for large teams, often requiring them to manage multiple caches across projects, generate and distribute static credentials, and implement complex access controls on their own.
As organizations grow, a number of problems compound:
* Specific team members need access to specific subsets of [flake outputs][outputs] and artifacts
* CI systems require secure, automated authentication
* Security teams need audit trails and compliance controls
* Using multiple caches can fragment build processes and slow down development
FlakeHub transforms this experience by providing a cache designed for teams.
Instead of managing multiple caches with static tokens, FlakeHub Cache provides a secure, unified, identity-aware cache with fine-grained access controls that integrates with your [existing identity management systems](#authentication).
## Private flakes
We're also excited to announce that [private flakes][private-flakes] are now generally available.
Private flakes enable teams to securely share and reuse Nix expressions without exposing sensitive code or configurations.
As with FlakeHub Cache, private flakes integrate seamlessly with your organization's existing [authentication flows](#authentication).
Like all flakes on FlakeHub, you can only [publish private flakes][publishing] from trusted CI systems (which means no ad-hoc publishing).
That currently includes [GitHub Actions][gh-actions] and [GitLab CI][gitlab-ci], but we plan to support other providers, like [CircleCI] and [Semaphore CI][semaphore], in the future.
To publish a private flake on [GitHub Actions][gh-actions], for example, you can use the [flakehub-push] Action, set the [visibility] to `private`, and you're good to go:
```yaml {3} title=".github/workflows/publish.yml"
- uses: DeterminateSystems/flakehub-push@main
with:
visibility: private
```
You can also use our [publishing wizard][wizard] to set up your Actions workflow.
## Federated authentication that makes sense \{#authentication}
FlakeHub Cache and private flakes are, naturally, features that require authentication.
But rather than relying on brittle auth patterns like long-lived static credentials, FlakeHub uses [JSON Web Tokens][jwt] (JWTs) to dynamically integrate with your existing identity infrastructure:
* [GitHub Actions][gh-actions] and [GitHub Enterprise OIDC][gh-enterprise]
* [GitLab CI/CD][gitlab-ci] for both cloud and self-hosted instances
* [Microsoft Entra ID][entra] SSO (formerly Azure Active Directory)
* [Amazon IAM][iam] role-based access
* Short-lived JSON Web Tokens (JWTs) for automation
When a machine authenticates using one of these methods, FlakeHub automatically knows to which organization the machine belongs—and to which flakes and cache slices it has access.
This approach aligns with zero-trust security principles, significantly reducing the attack surface and lowering the risk of issues like accidental cache poisoning.
## Built for teams
We know that different teams need different levels of access, so we've built out a robust policy engine undergirding FlakeHub Cache and private flakes.
We have built internal features around this policy engine, like IP restrictions and granting individual devices [deploy-only access to specific flake outputs][deploy-only-outputs].
In the future, users will be able to write their own custom policies that let you:
* Grant precise access to [flakes] and [flake outputs][outputs] based on team roles
* Control artifact visibility between projects
* Implement geographic access restrictions
* Enforce compliance requirements
* Monitor and audit [cache](#flakehub-cache) usage
Unlike systems with simplistic, all-or-nothing access controls, FlakeHub's policy engine provides fine-grained cache access control, enabling organizations to grant scoped access to users, devices, and automated processes.
This level of control is desirable—if not make-or-break—in highly regulated environments, supporting cross-team isolation and even geo-embargoes.
## Built to integrate \{#integrate}
While FlakeHub works with any Nix installation, it truly shines when paired with [Determinate Nix][det-nix] as [`determinate-nixd`][dnixd] is designed with FlakeHub Cache and private flakes in mind, creating an experience that just works out of the box.
Team members can authenticate to FlakeHub using the [`determinate-nixd login`][login] command, and CI pipelines and machines can automatically authenticate using their existing identity via `determinate-nixd login {aws|github-action|gitlab-pipeline}` (depending on the platform).
The thoughtful integration between Determinate Nix and FlakeHub delivers an experience where builds are faster, security is stronger, and developers can focus on their work instead of managing Nix infrastructure.
It's a practical demonstration of our vision for bringing Nix to professional teams, where every component is designed to work together seamlessly.
## Lightning-fast deployments
FlakeHub also enables you to rapidly deploy fully evaluated [closures].
You can quickly deploy [NixOS] configurations, [nix-darwin] configurations, and [Home Manager][hm] environments, without needing Nix to be installed on the target system, using [fh], the CLI for FlakeHub.
This command would apply a [NixOS] configuration to the current machine:
```shell title="Using fh apply to deploy a NixOS system"
fh apply nixos AmbiguousTechnologies/ethercalc/0.1#nixosConfigurations.ethercalc
```
And this command would apply a [nix-darwin] configuration to the current machine:
```shell title="Using fh apply to deploy a nix-darwin system"
fh apply nix-darwin AmbiguousTechnologies/ethercalc/0.1#darwinConfigurations.developer-workstation
```
Finally, this command would apply a [Home Manager][hm] configuration:
```shell title="Using fh apply to deploy a Home Manager configuration"
fh apply home-manager AmbiguousTechnologies/profile/0.1#homeConfigurations.developer-environment
```
In both cases, fh fetches the closure associated with the resolved store path *directly* from FlakeHub Cache; it does *not* [realise][realisation] any derivation or even evaluate the referenced expression.
That's because FlakeHub can **pre-generate store paths for you** and transfer computational responsibility from Nix to FlakeHub, which is particularly useful for deploying to resource-constrained devices.
We'll say much more about this—and provide a powerful demo—in an upcoming post.
And even though the `AmbiguousTechnologies/ethercalc` flake is [private](#private-flakes), this all works seamlessly as long as the machine is logged in using [`determinate-nixd login`](#integrate).
The current machine has access to the flake and to the respective cached outputs; non-logged-in machines have access to neither.
## Complete package independence
We were recently asked how FlakeHub Cache handles dependencies from [Nixpkgs] and other sources.
FlakeHub maintains complete copies of all packages, including those available from [cache.nixos.org][cno].
This design choice serves two key purposes:
1. It respects the Nix community's shared resources.
While we could save costs by deferring to [cache.nixos.org][cno], the upstream cache is a commons, and it isn't appropriate or respectful of those commons to save a few pennies in storage and bandwidth at the expense of the community project.
2. It ensures total reliability for your builds.
The upstream cache can be garbage collected or potentially tampered with, which could break or compromise your builds.
With FlakeHub Cache, your artifacts remain available for as long as you need them, independent of upstream availability.
## Available today
FlakeHub Cache and private flakes are [available now][pricing] for all FlakeHub organizations.
Pricing is straightforward: **$20** per FlakeHub organization member per month.
For a limited introductory period, all storage and bandwidth costs are included at no additional charge to help teams get started, and then priced at cost after that.
## The road ahead
FlakeHub Cache and private flakes are central to our mission to broaden Nix adoption, and this is just the start.
We have more to come in the not-too-distant future, so [stay tuned][blog].
We are eager to learn how teams utilize these features to enhance their development workflows while ensuring compliance and adopting a strong security posture.
Sign up now at https://flakehub.com and visit our [getting started docs][get-started] to begin taking advantage of these features in your workflows.
And as always, we'd love to hear your feedback on our Discord at https://determinate.systems/discord.
[all-flakes]: https://flakehub.com/flakes
[beta]: /blog/flakehub-cache-beta
[blog]: /blog/categories/announcement
[cache-docs]: https://docs.determinate.systems/flakehub/cache
[caching]: https://zero-to-nix.com/concepts/caching
[circleci]: https://circleci.com
[closures]: https://zero-to-nix.com/concepts/closures
[cno]: https://cache.nixos.org
[colmena]: https://flakehub.com/flake/zhaofengli/colmena
[deploy-only-outputs]: https://docs.determinate.systems/guides/resolve-only-deploy
[det-nix]: /blog/announcing-determinate-nix
[discovery]: https://docs.determinate.systems/flakehub/discovery
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[entra]: https://www.microsoft.com/security/business/identity-access/microsoft-entra-id
[fh]: https://github.com/DeterminateSystems/fh
[flakehub-announcement]: /blog/introducing-flakehub
[flakehub-push]: https://github.com/DeterminateSystems/flakehub-push
[flakes]: https://zero-to-nix.com/concepts/flakes
[get-started]: https://docs.determinate.systems/getting-started
[gh-actions]: https://github.com/features/actions
[gh-enterprise]: https://docs.github.com/en/enterprise-cloud@latest/admin/managing-iam/configuring-authentication-for-enterprise-managed-users/configuring-oidc-for-enterprise-managed-users
[gitlab-ci]: https://docs.gitlab.com/ee/ci
[hm]: https://github.com/nix-community/home-manager
[iam]: https://aws.amazon.com/iam
[jwt]: https://jwt.io
[login]: https://docs.determinate.systems/determinate-nix/#determinate-nixd-login
[nix]: /nix
[nix-darwin]: https://github.com/LnL7/nix-darwin
[nixos]: https://zero-to-nix.com/concepts/nixos
[nixpkgs]: https://github.com/NixOS/nixpkgs
[outputs]: https://zero-to-nix.com/concepts/flakes#outputs
[pricing]: https://flakehub.com/pricing
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[publishing]: https://docs.determinate.systems/flakehub/publishing
[realisation]: https://zero-to-nix.com/concepts/realisation
[semaphore]: https://semaphore.io
[signup]: https://flakehub.com/login
[semver]: https://docs.determinate.systems/flakehub/discovery
[tailscale]: https://flakehub.com/flake/tailscale/tailscale
[visibility]: https://docs.determinate.systems/flakehub/concepts/visibility#private
[wizard]: https://flakehub.com/new
---
**Announcing Determinate Nix**
Published: October 21, 2024
URL: https://determinate.systems/blog/announcing-determinate-nix
Today, I'm excited to announce [**Determinate Nix**](/nix), [Determinate Systems][detsys]' distribution of [Nix] built for teams and optimized for the enterprise.
Nix is extremely versatile and powerful, but as is often the case with free software projects, it is also unopinionated and "low policy."
As a result, getting Nix to work well in a development team requires a frustrating amount of configuration.
For instance:
* Setting up access to a private binary cache requires figuring out access control, distributing credentials, and configuring files like [`nix.conf`][nix-conf].
* While [Determinate Nix Installer][installer] has greatly improved the installation experience on [macOS], enterprise users still lack features like Mobile Device Management (MDM) support, proper [Amazon EC2][ec2] integration and [Keychain] support.
* Nix doesn't enable garbage collection by default, so users' disks have a tendency to fill up.
With Determinate Nix, our goal is to transform Nix from what it is today—a tool with great potential but with too many hard edges to be ready for prime time—into a part of your stack that does the Right Thing *out of the box* for teams big and small.
To begin making that a reality, the initial launch includes:
* Automatic **binary cache configuration**: to get access to your organization's [FlakeHub binary cache][cache], you just need to run `determinate-nixd login` (more on Determinate Nixd below).
* A much-improved experience on [**macOS**][macos]. For instance, Determinate Nix automatically uses certificates from the macOS [Keychain], and it supports [fully automated installation on AWS EC2 instances][macos-ec2].
* Periodic **garbage collection** quietly guards your system against Nix store bloat.
* **Amazon Web Services integration**: Determinate Nix can automatically log in to FlakeHub using [AWS IAM][iam] roles.
Determinate Nix is *not* a fork of Nix—it is a downstream Nix distribution.
Its features are implemented through a separate daemon called [Determinate Nixd][dnixd].
It's written in [Rust] (for the sake of memory safety) and it supervises the regular [Nix daemon][nix-daemon] while also providing some [other utilities][dnixd].
Our Nix distribution is carefully vetted to ensure compatibility and stability, guided by the telemetry collected by our [Determinate Nix Installer GitHub Action][installer-action].
## The big picture
Determinate Nix is part of a broader product experience that we call **Determinate**, which you'll be hearing much more about in the coming days.
Our goal for Determinate is to enable fearless innovation by bringing Nix to teams, providing a complete Nix-based workflow from installation through collaboration and CI to deployment.
The other central component of Determinate is [**FlakeHub**][flakehub], a service that provides a place for teams to [privately][private-flakes] publish flakes.
It provides a binary cache called [**FlakeHub Cache**][cache] that supports fine-grained access control policies as well as support for [private flakes][private-flakes].
But to use FlakeHub and FlakeHub Cache on developer workstations and in CI requires a fair amount of error-prone configuration when you're using regular Nix.
So one of the main reasons why we created Determinate Nix is to make Nix "just work" with the Determinate platform.
We'll talk more about private flakes, binary cache, and the Determinate big picture in upcoming blog posts!
## Future plans
We will continue to add new features to Determinate Nix to make the Nix user experience ever smoother for teams.
These include better authentication support for flakes and binary caches, [flake schemas][schemas], [parallel evaluation][parallel], and much more.
## Getting Determinate Nix
If you're using the Determinate Nix Installer, then getting Determinate Nix is as straightforward as adding the `--determinate` flag to the installation command:
```shell title="One-liner for installing Determinate Nix"
curl --proto '=https' --tlsv1.2 -sSf -L \
https://install.determinate.systems/nix | \
sh -s -- install --determinate
```
For NixOS users, we provide a [flake][determinate-flake] that makes switching to Determinate Nix straightforward.
For more information on installation and use, see the [Determinate documentation][getting-started].
We're interested in your feedback and would love to hear from you on our Discord at https://determinate.systems/discord.
[cache]: https://docs.determinate.systems/flakehub/cache
[determinate-flake]: https://flakehub.com/flake/DeterminateSystems/determinate
[detsys]: https://determinate.systems
[dnixd]: https://docs.determinate.systems/determinate-nix#determinate-nixd
[ec2]: https://aws.amazon.com/ec2
[flakehub]: https://flakehub.com
[getting-started]: https://docs.determinate.systems/getting-started
[iam]: https://aws.amazon.com/iam
[installer]: https://github.com/DeterminateSystems/nix-installer
[installer-action]: https://github.com/DeterminateSystems/determinate-nix-action
[keychain]: https://developer.apple.com/documentation/security/keychain-services
[macos]: /blog/tags/macos
[macos-ec2]: /blog/unattended-nix-install-macos-aws-ec2
[nix]: https://zero-to-nix.com/concepts/nix
[nix-conf]: https://nix.dev/manual/nix/latest/command-ref/conf-file
[nix-daemon]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-daemon
[parallel]: /blog/parallel-nix-eval
[private-flakes]: https://docs.determinate.systems/flakehub/private-flakes
[rust]: https://rust-lang.org
[schemas]: /blog/flake-schemas
---
**Discontinuing support for macOS Monterey**
Published: October 9, 2024
URL: https://determinate.systems/blog/nix-installer-macos-12-monterey
Determinate Nix Installer and related software will no longer support macOS Monterey (version 12).
This change follows Apples' standard procedure of discontinuing support for older macOS versions.
### Timeline
Our software will no longer explicitly target or test against macOS Monterey starting **Monday, October 21, 2024**.
Our deprecation date is in advance of [GitHub's own deprecation timeline][github], which drops support on December 3, 2024.
Note that between October 21 and December 3, [`DeterminateSystems/nix-installer-action`][nix-installer-action] and [`DeterminateSystems/magic-nix-cache-action`][magic-nix-cache-action] will detect macOS Monterey, and fall back to the last release known to support macOS Monterey. Security patches will not be back ported to these older releases. This fallback behavior will be removed after GitHub fully disables hosted runners for macOS Monterey.
### Affected Software
* nix-installer
* fh
* flake-checker
* flakehub-push
* determinate-nixd
* magic-nix-cache
### How will this affect users?
According to our data, most macOS Monterey users are on GitHub Actions.
These users will need to update their workflows to use a more recent macOS version, like `macos-latest`.
But we understand that this change might impact macOS Monterey users we're unaware of, as future updates to our software will no longer explicitly support this platform.
We're here to help.
If you do need continued support for macOS Monterey, please reach out to us at hello@determinate.systems.
Our goal is to ensure that your projects continue to run smoothly and we're open to finding ways to accommodate your requirements.
We would be happy to discuss your needs and explore potential solutions.
[github]: https://github.com/actions/runner-images/issues/10721
[nix-installer-action]: https://github.com/DeterminateSystems/nix-installer-action
[magic-nix-cache-action]: https://github.com/DeterminateSystems/magic-nix-cache-action
---
**Fully automated Nix installation for macOS on AWS EC2**
Published: October 1, 2024
URL: https://determinate.systems/blog/unattended-nix-install-macos-aws-ec2
Installing Nix on macOS has been [largely solved][announcement-determinate-nix-installer] for some time now, but a quirk of [Amazon Web Services][aws]' macOS support [created a huge wrinkle][upstream-bug] for those running macOS on AWS.
This quirk meant that users needed to graphically log in over VNC to manually approve "full disk access" to the Nix daemon.
This wrinkle is now smoothed out, and the installation is fully automatable.
Users of the [Determinate Nix Installer][installer] can now install Nix to macOS on Amazon Web Services without needing to interact with any graphical user interface.
## Unlocking new use cases like autoscaling macOS
Previously, Nix users couldn't autoscale macOS on AWS—after all, there's nothing "auto" about manually approving full disk access!
With automated Nix installation, autoscaling is seamless, allowing for larger and more flexible AWS deployments.
### Appropriate use cases and limitations
The new fully unattended installation isn't appropriate for all use cases, as this new behavior brings some limitations to your instance lifecycle.
| Use case | Status |
|-------------------------------------------------------------------------------|-------------------|
| Ephemeral macOS instances that terminate when the machine is no longer needed | Fully automatable |
| Auto-scaled macOS infrastructure | Fully automatable |
| Long-term macOS instances that are stopped and re-started | Do not automate |
| macOS instances that are snapshotted and cloned | Do not automate |
See the caveats [below](#caveats).
## How to install Nix on macOS in EC2
Run this install command:
```shell title="Install Nix" {4,5}
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix \
| sh -s -- install macos \
--no-confirm \
--determinate \
--use-ec2-instance-store
```
## Why is macOS on AWS different?
Amazon's Apple hardware boots macOS from an EBS volume which is presented over PCIe.
Because this volume isn't the soldered-in hardware, macOS considers it to be a removable volume.
macOS' permissions model requires software accessing removable volumes to be granted special privileges.
The Determinate Nix Installer creates a new volume for the Nix Store on the same disk as the operating system.
That new volume is considered "removable" even though it is on the same disk.
## How does it work?
The core change we've made is that we've added a `--use-ec2-instance-store` flag for installing Determinate Nix.
When this flag is set, the installer installs Nix to a volume on the internal disk, eliminating the need for manual approval.
Note that this feature is limited to Determinate users (`--determinate`) due to runtime orchestration provided by `determinate-nixd`.
### Caveats
Setting the `--use-ec2-instance-store` flag installs Nix to the instance's ephemeral instance store.
Using the instance store means that:
* The Nix Store is erased when the machine is stopped.
* The Nix Store is not captured by EBS snapshots.
* Standard macOS reboots are perfectly safe.
Please see [Data persistence for Amazon EC2 instance store volumes][data-persistence] for further details.
## Ready to automate your macOS CI pipeline?
We've built a comprehensive guide that takes this announcement to the next level, with step-by-step instructions for setting up auto-scaling macOS CI infrastructure with AWS IAM authentication.
Learn how to:
* Deploy with EC2 launch templates
* Configure secure IAM authentication with FlakeHub
* Set up auto-scaling groups for elastic CI capacity
* Integrate with AWS Systems Manager for management
**[Check out our detailed deployment guide →](https://docs.determinate.systems/guides/deploy-nix-macos-ec2)**
Join the teams already using Determinate Nix on AWS to build faster, more secure CI pipelines.
Have questions? Join us on [Discord](https://determinate.systems/discord) where our team and community can help you get started.
[announcement-determinate-nix-installer]: /blog/determinate-nix-installer/
[aws]: https://aws.amazon.com
[data-persistence]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-store-lifetime.html
[installer]: https://github.com/DeterminateSystems/nix-installer
[upstream-bug]: https://github.com/NixOS/nix/issues/6291
---
**Solving corporate TLS certificates for Nix on macOS**
Published: September 27, 2024
URL: https://determinate.systems/blog/zscaler-macos-and-nix-on-corporate-networks
Enterprises often use tools like [Zscaler] to bolster their overall security posture.
Zscaler is an intercepting TLS proxy that inspects the traffic coming in and going out of the network.
Because it intercepts *all* traffic, clients have to be configured with a custom TLS certificate for the connections to be trusted.
This doesn't always play nicely with other tools, especially tools that require TLS certificates—like [Nix].
## Zscaler and Nix used to be a big pain
Nix users in enterprise environments have run head-first into this problem countless times:
```log
warning: unable to download '...': SSL peer certificate or SSH remote key was not OK (60)
```
On Linux, this isn't a huge issue.
Users and admins can add the certificate to their trusted bundle at a standard location like `/etc/ssl/certs/ca-certificates.crt` or `/etc/pki/tls/certs/ca-bundle.crt`.
On macOS, however, the story is slightly more annoying for Nix because macOS stores the custom certificates in [Keychain], a key and secret store used by applications across the macOS ecosystem.
If you're fully steeped in macOS, this is fine and works out of the box.
But if you're using a tool like Nix that's mostly designed and used by Linux users to build Linux software then... things can get unpleasant.
Users have long had to export their enterprise certificate from Keychain, reconfigure Nix, and manage this over time.
When the certificate eventually expires and needs rotating, the user has to figure out how they fixed it the first time and try again.
{/* vale off */}
*Uff da*... what a headache.
{/* vale on */}
## Determinate means Nix + Zscaler doesn't have to be a huge pain
Our goal with [Determinate] is to make Nix a pleasure to use across these platforms and to elegantly solve thorny problems for both users and IT administrators.
So we fixed the Nix/Zscaler certificate issue on macOS.
Determinate automatically configures Nix on macOS with an up-to-date certificate bundle from Keychain and synchronizes the bundle with Keychain over time.
## How to install Determinate
Install Determinate with the [Determinate Nix Installer][installer] by passing the `--determinate` flag:
```shell
curl --proto '=https' --tlsv1.2 -sSf -L \
https://install.determinate.systems/nix | sh -s -- install --determinate
```
Alternatively, you can use our [signed macOS `.pkg`][macos-pkg] for convenient distribution.
If [Installomator] is your thing, we provide a [Mobile Device Management (MDM) script][mdm-script] that integrates especially well with automated processes.
Determinate is available on all systems that the Determinate Nix Installer supports.
## More to come
This is just the beginning of a series of posts about what we're doing to improve the experience of using Nix in the enterprise.
If this is music to your ears, join our Discord at https://determinate.systems/discord and come chat.
[determinate]: /nix
[Installomator]: https://github.com/Installomator/Installomator
[installer]: https://github.com/DeterminateSystems/nix-installer
[keychain]: https://support.apple.com/guide/keychain-access/welcome/mac
[macos-pkg]: https://docs.determinate.systems/getting-started/#determinate-pkg
[mdm-script]: https://docs.determinate.systems/advanced/deploy-with-mdm
[nix]: https://zero-to-nix.com/concepts/nix
[zscaler]: https://zscaler.com
---
**Prepare Nix for macOS Sequoia**
Published: September 13, 2024
URL: https://determinate.systems/blog/nix-support-for-macos-sequoia
Determinate Nix Installer v0.26.0 has been released, including a `repair sequoia` command that addresses issues with the upgrade path to macOS Sequoia.
Before upgrading to Sequoia, Nix users should run the repair tool:
```
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix/tag/v0.26.0 | sh -s -- repair sequoia --move-existing-users
```
Running the repair tool after upgrading to Sequoia will work, too, but Nix will not work until after the repair tool is run.
### Who needs to run the repair tool?
Nix users on macOS who installed Nix with the upstream Nix installer, or Determinate Nix Installer [`v0.22.0`](https://github.com/DeterminateSystems/nix-installer/releases/tag/v0.22.0) and older.
Run `/nix/nix-installer --version` to identify the Determinate Nix Installer version used on your system.
The repair tool is safe to run, even if you installed with a Determinate Nix Installer version newer than [`v0.22.0`](https://github.com/DeterminateSystems/nix-installer/releases/tag/v0.22.0).
### Can I repair my Nix installation that wasn't installed by the Determinate Nix Installer?
Yes.
The Determinate Nix Installer's repair tool works on any Nix installation on macOS.
#### Caveat for advanced users
Users who customized their Nix build users should read the help output of the `repair sequoia` command by passing the `--help` flag.
### Why is this repair tool required?
macOS Sequoia creates users with `UniqueID`s that conflict with Nix's `_nixbldN` (that is `_nixbld1`) users.
When upgrading to Sequoia, macOS' update tool deletes the conflicting users.
### More information
For more information, see the upstream Nix tracking issue: https://github.com/NixOS/nix/issues/10892
Or, join our [Discord](https://determinate.systems/discord) and chat with the team.
---
**Discontinuing support for i686-linux in the Determinate Nix Installer**
Published: July 24, 2024
URL: https://determinate.systems/blog/nix-installer-i686-linux
Here at [Determinate Systems][detsys], our mission is to transform the software industry by bringing [Nix] to the enterprise.
As a small team, we periodically evaluate how we spend our time and the impact that we have on our users.
Today, we're announcing that we plan to discontinue supporting `i686-linux` in the [Determinate Nix Installer][installer].
The decision to end support for `i686-linux` comes after carefully reviewing usage patterns and the resources required to maintain this system type.
For `i686-linux`, we saw only *one* attempt to install Nix in the last three months—and that attempt failed.
Given this essentially non-existent usage, the burden of continuing to support and test compatibility significantly outweighs the benefits.
### Timeline
Determinate Nix Installer releases will continue to support `i686-linux` until **August 15, 2024**.
### How will this affect users?
According to our data, **it won't**.
But we understand that this change might impact `i686-linux` users we're unaware of, as future updates to the Determinate Nix Installer will no longer support this architecture.
We're here to help.
If you do need continued support for `i686-linux`, please reach out to us at hello@determinate.systems.
Our goal is to ensure that your projects continue to run smoothly and we're open to finding ways to accommodate your requirements.
We would be happy to discuss your needs and explore potential solutions.
The [Determinate Nix Installer][installer] will continue to support these systems for the foreseeable future:
* `x86_64-linux`
* `aarch64-linux`
* `x86_64-darwin`
* `aarch64-darwin`
[detsys]: /
[installer]: https://github.com/DeterminateSystems/nix-installer
[nix]: https://zero-to-nix.com/concepts/nix
---
**Parallel Nix evaluation**
Published: June 27, 2024
URL: https://determinate.systems/blog/parallel-nix-eval
The Nix user experience is significantly affected by the speed of the [Nix] expression evaluator.
[Nixpkgs] and [NixOS] have grown massively in recent years, and continue to grow.
This means that operations like [`nix search nixpkgs`][search] or evaluating NixOS system configurations take increasing amounts of time.
For instance, listing all packages (`nix-env -qa`) took less than 3 seconds in Nixpkgs 16.03, but more than 15 seconds in 24.05; evaluating a small NixOS configuration (`closures.lapp.x86_64-linux`) finished in less than half a second in 16.03 but needs more than 3 seconds today.
Similarly, many users have large [flakes] that take a long time to evaluate, either in CI or on their own systems (for example when running [`nix flake check`][nix-flake-check]).
Many improvements have been made to the Nix evaluator in recent times, but one obvious avenue for improvement has not been taken so far: *parallel evaluation*.
The Nix evaluator has always been single threaded, which means that Nix hasn't been able to utilize all those wonderful CPU cores available on modern systems—yet.
At [Determinate Systems](/) we've recently implemented a parallel evaluator, currently available in [a draft PR][draft].
It already gives a substantial performance improvement—speedups of a factor 3-4 are typical.
In this blog post, we describe some of the work that was needed to achieve parallelism, preceded by a short overview of the inner workings of the evaluator.
## How Nix evaluation works
In the Nix evaluator, every value is represented as a 24-byte (3-word) structure that consists of a *type* field and a type-dependent *payload*.
The type is a tag such as "integer", "Boolean" or "lambda."
For example, the attribute
```nix
x = 2;
```
produces the value
In principle, purely functional languages like Nix are ideal for parallel evaluation, because values are immutable.
This means that the pointer to the value of `x` can be freely shared between threads, as we don't have to worry about the value getting updated by one thread while another is accessing it.
There is one exception, however, to the immutability of values: *thunks*.
These are the mechanism by which lazy evaluation is implemented.
A thunk represents a delayed computation, or *closure*, that is *overwritten in place* by the result of the computation, if and when the result is needed.
A thunk consists of two pieces of information:
* A pointer to an abstract syntax representation of the definition of the value (in compiled languages, this would be a pointer to the compiled code, but the Nix evaluator is an *interpreter*, not a compiler).
* A pointer to the *environment*, which is a data structure that records all the variables that are in scope.
In an expression like `let x = 1; in let y = 2; in `, for instance, the environment of the thunk for expression `` maps the variables `x` and `y` to their respective values.
As an example, consider the expression
```nix
let
x = 2;
y = 3;
z = x * y;
in z
```
The value `z` is initially represented as a thunk:
If `z` is never needed, then `z` will forever remain a thunk.
But if another expression needs `z`, the thunk will be *forced*, meaning that the expression `x * y` will be computed using the environment that maps `x` to 2 and `y` to 3, and thunk is overwritten:
Any subsequent use of `z` will see the final value `6`, so `z` will be evaluated at most once.
## Making the evaluator thread safe
To speed up operations like `nix search nixpkgs`, we would like to distribute the work of evaluating the attributes in Nixpkgs across multiple threads.
As noted above, only thunks represent a difficulty; "final" values like integers or attribute sets can be accessed in parallel since they're never updated.
To handle thunks correctly and efficiently, we have these requirements:
* No thread should ever see a "half-updated" thunk, for example one where the "type" field has been changed to "integer" but the integer payload is in an undefined state.
* A thunk should not be evaluated more than once.
Thus, if a thunk is being evaluated by one thread, other threads that need the result of the thunk should wait (or do other work).
* Support for multi-threaded evaluation should not have a negative impact on single-threaded performance or memory consumption.
This means, for instance, that we can't just wrap every value in a [mutex].
In addition, the changes should be minimally invasive to the current evaluator.
The approach we took is fairly straightforward:
* The "type" field in values is now an [`std::atomic`][atomic].
This allows its state transitions to be done using atomic compare-and-swap instructions, and enables the necessary compiler and CPU memory barriers that ensure that threads see changes to values in the right order.
* The payload of a thunk value is updated to its final value *before* the type is updated.
Thus, if a thread sees a value with an immutable type (for example "integer"), then its payload (for example the integer value) is also valid.
* There are three new value types:
* **Pending** denotes a thunk that is being evaluated by a thread, and has no other threads waiting for it.
* **Awaited** denotes a thunk that is being evaluated by a thread, and has other threads waiting for it.
* **Failed** denotes a thunk whose evaluation threw an exception.
Its payload stores an [`std::exception_ptr`][ptr] that allows other threads to re-throw the exception if needed.
* The lifecycle of a thunk is as follows:
* The value is initialized as a thunk.
* When a thread forces a value that has type "thunk", it sets its type to "pending".
This is done using an atomic compare-and-swap to ensure that only one thread will actually evaluate the thunk.
* When a thread forces a value that has type "pending", it registers itself in a list of waiters (stored separately from the value) and sets the type of the value to "awaited" (using an atomic compare-and-swap to handle the case where the first thread has just finished the value).
It then waits to be woken up by the first thread when it has finished with the value.
* When a thread finishes evaluating a thunk, it updates the payload of the value, and then does an atomic swap to change the type to its final type.
If the previous type was "pending", that's all it needs to do, since no other threads are waiting. If it was "awaited", it wakes up the threads that are waiting.
Beyond thunk handling, a couple of other data structures in the evaluator had to be made thread-safe.
Usually it was sufficient to wrap them in a mutex, but some data structures are highly contended and so required special handling to make them performant.
In particular the *symbol table*, which ensures that every identifier is stored in memory only once, was rewritten to make symbol access lock-free.
## Results
Currently only two `nix` subcommands distribute work across multiple threads.
This command shows the contents of the `nix` flake:
```bash
nix flake show --no-eval-cache --all-systems --json \
github:NixOS/nix/afdd12be5e19c0001ff3297dea544301108d298
```
On a Ryzen 5900X (12 cores), the single-threaded evaluator takes **23.70s**.
The multi-threaded evaluator brings this down to **5.77s** (using 12 threads), a **4.1x** speedup.
This command searches the contents of Nixpkgs:
```bash
nix search --no-eval-cache github:NixOS/nixpkgs/bf8462aeba50cc753971480f613fbae0747cffc0 ^
```
This went from **11.82s** to **3.88s** (using 16 threads), a **3.0x** speedup.
This is less than for the previous benchmark because there are more shared dependencies (for example, most threads cannot make progress until the `stdenv` value has been computed).
## Trying it out
The multi-threaded evaluator still has a couple of concurrency bugs that need to be sorted out, but if you do want to try it out, you can get it by running:
```bash
nix shell github:DeterminateSystems/nix-src/multithreaded-eval
```
You then need to set the environment variable `NR_CORES` to the number of threads that Nix should use.
Here's an example:
```bash
NR_CORES=8 nix search --no-eval-cache nixpkgs hello
```
## Next steps
In the near future, we will make more Nix subcommands multi-threaded, such as [`nix flake check`][nix-flake-check].
In addition, evaluations of single-flake output attributes such as NixOS system configurations should exploit parallelism automatically.
One way to do this would be to distribute derivation attributes across threads.
[atomic]: https://en.cppreference.com/w/cpp/atomic/atomic
[draft]: https://github.com/NixOS/nix/pull/10938
[flakes]: https://zero-to-nix.com/concepts/flakes
[mutex]: https://en.wikipedia.org/wiki/Lock_(computer_science)
[nix]: https://zero-to-nix.com/concepts/nix
[nix-flake-check]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-flake-check
[nixos]: https://zero-to-nix.com/concepts/nixos
[nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs
[ptr]: https://en.cppreference.com/w/cpp/error/exception_ptr
[search]: https://zero-to-nix.com/start/nix-search
---
**Nix as a WebAssembly build tool**
Published: June 13, 2024
URL: https://determinate.systems/blog/nix-wasm
I've been pretty bullish on [**WebAssembly**][wasm]—or **Wasm**—for quite some time, as I believe that it offers a degree of portability and operational simplicity that goes beyond that of [Linux virtual machines][vms] and even [OCI containers][containers].
Run it in the browser, run it on your laptop, run it on [Kubernetes][k8s], run it on a dedicated Wasm platform like [Fermyon].
While I'm not convinced that it will fully *supplant* VMs or containers any time soon, I do think that there's a strong case that Wasm is already a superior technology in domains like [edge functions][edge] and [platform extensions][extensions].
But alas, there's a bit of a snag: Wasm is extremely portable *once you've already built it*.
But building Wasm isn't trivial for three reasons:
* You can compile many languages to Wasm and each has its own tools and approaches.
* There are numerous Wasm-specific tools that you may want to include in your toolchain, such as [wasm-tools] and the [WebAssembly Binary Toolkit][wabt] (WABT).
* There are several Wasm runtimes currently available, including [WasmEdge] and [Wasmtime].
And any time you need a bunch of separate tools to get your work done, there's room for error and the classic "read the README and use apt-get/Homebrew/whatever to create your environment" approach to dependency management quickly runs into hard limits.
Unsurprisingly, I think that using [Nix] to package Wasm provides a compelling path forward here.
## My example project
To show how you can use Nix to work with Wasm, I've created an example project in the [DeterminateSystems/nix-wasm-example][example] repo.
The actual [Wasm app][app] I build here is extremely basic: just a [Rust] program that outputs the string `"Hello there, Nix enthusiast!"`
What's notable here is that the program is compiled to conform to the [WebAssembly System Interface][wasi] (WASI), which essentially means that it's built to interact with the outside world (by default, WebAssembly can't act as a command line interface or system tool).
To build the program into WASI-compliant Wasm, I created a special Nix function called [`buildRustWasiWasm`][build-wasi] that wraps [Naersk]'s `buildPackage` derivation function.
That function builds the Rust sources using a special Rust toolchain that includes the [`wasm32-wasip1`][target] target.
The post-install phase of the derivation also uses [`wasm-strip`][wasm-strip] to make the final Wasm binary more lean and [`wasm-validate`][wasm-validate] to ensure that the resulting binary is valid.
```shell title="Build hello-wasm and inspect the result"
nix build "https://flakehub.com/f/DeterminateSystems/nix-wasm-example/*.tar.gz#hello-wasm"
ls ./result/lib
```
You should see a `hello-wasm.wasm` binary in that directory.
Being able to build WebAssembly from Rust sources deterministically, without needing to use `apt-get` or Homebrew or anything else, is nice.
But Nix enables you to do much more, so let's have a bit more fun.
## A full Wasm package
The `nix build` command above deterministically builds a single Wasm binary from Rust source.
Build this derivation and inspect the output:
```shell title="Build hello-wasm-pkg and inspect the result"
nix build "https://flakehub.com/f/DeterminateSystems/nix-wasm-example/*.tar.gz#hello-wasm-pkg"
tree result
```
That yields this filesystem tree:
```shell title="Filesystem tree for ./result"
result
├── lib
│ └── hello-wasm.wasm
└── share
├── hello-wasm-dump.txt
├── hello-wasm.dist
└── hello-wasm.wat
3 directories, 4 files
```
What you see here is a kind of WebAssembly package that includes not just the executable Wasm binary but also some information about it:
* The `hello-wasm-dump.txt` file is produced by the [wasm-objdump] tool.
It provides information about our binary, including headers, type definitions, function definitions, and more, which can be used by IDEs, debuggers, and other tools.
* The `hello-wasm.dist` file is produced by the [wasm-stats] tool.
It provides information about the size of sections, functions, and more, which can be used to optimize performance, to debug, and more.
* The `hello-wasm.wat` file is a human-readable textual representation of the binary, built by the [wasm2wat].
Tools like [wat2wasm] can use these files to generate Wasm from that textual format and runtimes like [Wasmtime] can run them directly.
This package is built using the [`buildRustWasmPackage`][build-wasm-pkg] function, which wraps the [`buildRustWasiWasm`][build-wasi] function mentioned above.
There are plenty of other things we could add to such a "Wasm package," but this provides a small taste.
## A working CLI
While you can run WebAssembly in the browser, in the cloud, on [Kubernetes][k8s], and in many other places, an emerging use case is running it as a CLI tool.
What makes using Wasm as a CLI tool a bit tricky on a lot of systems is that you need to have a Wasm runtime present on the system to convert WASI-compatible Wasm into system calls.
With Nix, we can directly solve this problem by creating derivations that use a Wasm runtime to run a compiled Wasm binary.
Let's run our compiled binary using the [Wasmtime] runtime:
```shell title="Run the compiled binary using Wasmtime"
nix run "https://flakehub.com/f/DeterminateSystems/nix-wasm-example/*.tar.gz#hello-wasmtime-exec"
```
Here, I created a [`buildRustWasmtimeExec`][build-wasmtime-exec] function that creates a wrapper script using [`makeWrapper`][wrapper] that runs [WasmEdge] and passes in a path to our compiled Wasm binary (all of this happen in the [Nix store][store], of course).
You can also run the binary using the [WasmEdge] runtime:
```shell title="Run the compiled binary using WasmEdge"
nix run "https://flakehub.com/f/DeterminateSystems/nix-wasm-example/*.tar.gz#hello-wasmedge-exec"
```
As with WasmTime, I created a [`buildRustWasmEdgeExec`][build-wasmedge-exec] function that creates a wrapper script that runs [WasmEdge] and passes in a path to our compiled Wasm binary.
Congrats! You just ran two compiled Wasm binaries on your machine using two separate Wasm runtimes, with everything built deterministically and reproducibly using Nix.
## Beyond containers and VMs
This example project is quite unambitious but I hope that it shows that Nix provides a wealth of possibilities in the WebAssembly domain (and other domains like it).
Wasm is not trivial to build and package and Nix is a far better tool for it than the usual Makefiles and Bash.
From a deployment perspective, Nix is typically seen as a tool that can build things like OCI containers or artifacts for running virtual machines (like [ISOs]).
But the case of Wasm shows that Nix would be indispensable even in some future world where our industry has gone all-in on Wasm and moved beyond both containers and VMs.
[app]: https://github.com/DeterminateSystems/nix-wasm-example/tree/main/src/main.rs
[build-wasi]: https://github.com/DeterminateSystems/nix-wasm-example/blob/c69bd6eff6ba25ff9ba887d6688538c5279b4252/flake.nix#L97-L114
[build-wasm-pkg]: https://github.com/DeterminateSystems/nix-wasm-example/blob/c69bd6eff6ba25ff9ba887d6688538c5279b4252/flake.nix#L156-L180
[build-wasmedge-exec]: https://github.com/DeterminateSystems/nix-wasm-example/blob/c69bd6eff6ba25ff9ba887d6688538c5279b4252/flake.nix#L136-L153
[build-wasmtime-exec]: https://github.com/DeterminateSystems/nix-wasm-example/blob/c69bd6eff6ba25ff9ba887d6688538c5279b4252/flake.nix#L116-L134
[containers]: https://opencontainers.org
[edge]: https://en.wikipedia.org/wiki/Edge_computing
[example]: https://github.com/determinateSystems/nix-wasm-example
[fermyon]: https://fermyon.com
[extensions]: https://www.salaboy.com/2023/04/15/extending-platforms-with-webassembly
[isos]: https://docs.fileformat.com/compression/iso
[k8s]: https://kubernetes.io
[naersk]: https://github.com/nix-community/naersk
[nix]: https://zero-to-nix.com/concepts/nix
[rust]: https://rust-lang.org
[store]: https://zero-to-nix.com/concepts/nix-store
[target]: https://blog.rust-lang.org/2024/04/09/updates-to-rusts-wasi-targets.html
[vms]: https://www.vmware.com/topics/glossary/content/virtual-machine.html
[wabt]: https://github.com/WebAssembly/wabt
[wasi]: https://wasi.dev
[wasm]: https://webassembly.org
[wasm2wat]: https://webassembly.github.io/wabt/doc/wasm2wat.1.html
[wasmedge]: https://wasmedge.org
[wasmtime]: https://docs.wasmtime.dev
[wasm-objdump]: https://webassembly.github.io/wabt/doc/wasm-objdump.1.html
[wasm-stats]: https://webassembly.github.io/wabt/doc/wasm-stats.1.html
[wasm-strip]: https://webassembly.github.io/wabt/doc/wasm-strip.1.html
[wasm-tools]: https://github.com/bytecodealliance/wasm-tools
[wat2wasm]: https://webassembly.github.io/wabt/doc/wat2wasm.1.html
[wasm-validate]: https://webassembly.github.io/wabt/doc/wasm-validate.1.html
[wrapper]: https://nixos.org/manual/nixpkgs/stable/#fun-makeWrapper
---
**On community in Nix**
Published: April 26, 2024
URL: https://determinate.systems/blog/on-community-in-nix
Dear members of the Nix Community,
When I created the [Nix](https://nixos.org/) project as part of my PhD thesis research in 2003, I could never have imagined the enthusiasm that people would have for it, the [energetic community](https://nixos.org/community/) that would grow around it, and the myriad of amazing new things that people would use it to help bring to life.
I am immeasurably pleased and proud of where the project stands today and extremely optimistic for its longevity and utility as a critical component of the software delivery pipeline for an ever-growing number of developers, teams and businesses.
I co-created and assumed leadership of the NixOS Foundation in 2015 out of a sense of duty to further facilitate the rapid growth of the project and the community.
The Foundation had modest goals: it is intended to provide continuity for critical assets like the binary cache, and to provide legal and financial support to events like NixCon.
It was not intended to “run” the Nix/NixOS community, which has done an admirable job in managing itself without top-down control.
I am glad that I helped to establish the Foundation, and that it continues to serve the practical purposes that it should.
That said, my participation in the much-needed administrative work that the body does is not my passion and I would be happy to step away from it at some point in the future.
In July 2022, I co-founded Determinate Systems, a company that will accelerate innovation and adoption of Nix by simplifying its use and helping users get predictably great results with it.
The company, its mission, and my place in its continued development are my focus and passion and are the vehicles through which I am committed to continuing to contribute to the development of Nix and the realization of its goals.
In recent months, and more intensely in recent days, a number of issues have been broached by a small group within the community regarding the governance of the project, the foundation, and the community itself. The ongoing debate on these issues has been divisive to the community and deleterious to our collective goals.
I would like to provide clarification regarding a number of misperceptions that have been brought to light as a result of the collective discussion.
First, I have had rather little involvement in Nixpkgs and NixOS in recent years, having ceded their stewardship to other community members some time ago, and have no more influence over their direction than any other dedicated contributor.
Second, I am just one member of the five-member Nix team and hold no more formal authority than the others in determining the direction of the team.
Third, I have not been a member of the RFC Steering Committee since new members were elected in January 2024.
Fourth, the NixOS Foundation in no way controls or governs the Nix community, which has, since its inception, demonstrated its ability to self-govern well.
With regard to the most focal issue raised in recent discussions, that of who should or shouldn't be allowed to sponsor or participate in NixCon, I feel that the chief aim of the Nix project is to give developers the tools they need to do better work, faster.
I strongly believe that we should not exclude any company from contributing to, participating in, or utilizing the Nix project in any way.
The nature of open source software is such that anyone, working for any company or none, can contribute to and benefit from the work being done by the broader community.
That companies create products and technologies that some approve of and others disapprove of is a fact of life.
It is my opinion that it is not for us, as open source software developers, to decide whose views are valid and whose are not, and to allow or disallow project or conference participation as a result.
We should welcome all contributions that help the project grow and thrive.
This is a view that I have and will always hold, regardless of my company's involvement in the project.
Regarding alleged conflicts of interest, I want to be clear.
My role, participation, and focus on the good work being done at Determinate Systems have been public knowledge since the company's inception.
The assertion that Determinate Systems "owns" Nix or seeks to exert out-sized influence over the project, the community, or the foundation is patently false: I am the only member of the Nix team who works for Determinate Systems.
Last, I must express my deep disappointment and disbelief at the accusation of excluding people from minority or marginalized backgrounds.
As someone who highly values diversity and inclusion, this accusation is not only unfounded but also insulting.
Throughout my career, I have consistently supported work towards creating a welcoming and inclusive environment for all individuals, regardless of their background.
I continue to support initiatives to ensure that everyone feels valued and appreciated, and I have actively encouraged opportunities to amplify the voices of those who have been historically marginalized.
I remain committed to creating a community where everyone feels seen, heard, and valued, and I will not let unfounded accusations detract from this important work.
I encourage everyone reading this who feels that they have not been heard or feels displaced to join the Determinate Systems community as we continue working to make Nix as usable and as impactful as possible.
Our code of conduct is available [here](https://flakehub.com/policy/code-of-conduct), and you can join our Discord at https://determinate.systems/discord.
In closing, I want to thank everyone in the Nix community for their hard work and dedication to the project.
I have nothing but respect, admiration, and deep appreciation for all of you and the commitment and progress that you have made.
I am excited to see what the future holds for Nix, and I am committed to continuing to do my part to help it grow and thrive.
Sincerely,
Eelco Dolstra.
---
**Introducing FlakeHub Cache**
Published: March 13, 2024
URL: https://determinate.systems/blog/flakehub-cache-beta
[Binary caching][caching] is truly one of the most wonderful core features of [Nix].
It enables you to fetch the build results of [derivations] rather than building them locally, which makes just about everything you do with Nix—[development environments][devenv], [package builds][packages], continuous integrations runs—so much faster that Nix without caching is essentially a different, lesser tool.
But despite our love for Nix caching, we've believed that existing caching solutions are not suitable for secure, production use cases.
And so we've opted to build something better.
Today, we're excited to announce **FlakeHub Cache**, a powerful new caching concept from us at [Determinate Systems][detsys] that offers [robust per-flake access control](#access-control).
FlakeHub Cache is currently in *private beta* and we're actively iterating on the implementation and seeking design partners to try it out.
You can sign up for the beta right here and we'll let you know next steps soon:
## The core feature: granular access control \{#access-control}
Nix's official binary cache server, [`nix-serve`][nix-serve], has some major gaps that make it unsuitable for secure, production use cases.
Most importantly, `nix-serve` serves up a [Nix store][store] as a single monolithic cache, and *everything* in the cache is available to anyone with a public key.
This is an unacceptably fast-and-loose access model for most organizations.
With FlakeHub Cache's access control model, however, you can grant or deny read and write access at the [flake][flakes] level, which affords you substantially more control.
So what does this look like?
### Reading from FlakeHub Cache \{#reading}
Let's start with read access to pull from a flake's cache.
If your org has a flake called `security-mega-important`, for example, you can provide read access only to a small set of trusted users—or automated agents—in your org.
If, on the other hand, you have a flake called `shared` that's meant to be used by everyone, you can provide read access to everyone in your org.
### Writing to FlakeHub Cache \{#writing}
In terms of write access to push to a flake's cache, we've opted for a model centered around controlled, authenticated build environments.
FlakeHub Cache currently allows pushing to the cache [solely](#model) as part of [GitHub Actions][actions] runs (with [GitLab] support on the way).
This means that there is *no ad-hoc push access whatsoever*, so you can't accidentally push to a cache in a shell script or CLI command.
The granular and controlled nature of this model is far more suited for organizations with demanding security and other requirements, and a significant leap beyond existing solutions.
### Authentication model \{#auth}
In order to grant or deny access, of course, FlakeHub Cache needs to figure out who—or what—is making a request.
Authentication for FlakeHub Cache is based on [JSON Web Tokens][jwt] (JWTs).
We currently use [GitHub] as our JWT authentication provider but will be adding [GitLab] support soon.
In the future, this flexible model will enable us to support a wide variety of authentication solutions, including [SAML] and various [Single Sign-on][sso] (SSO) providers.
Static tokens are great for some use cases—and FlakeHub enables you to create such tokens in the UI—but we're opting to go beyond this model.
## How the cache itself works \{#how}
FlakeHub Cache is able to provide [granular access control](#access-control) because of instead of serving an entire [Nix store][store] as "the cache," FlakeHub Cache applies a **slice** abstraction, where a slice is some subset of all store paths.
With FlakeHub Cache enabled, each flake you publish on [FlakeHub] gets its own slice and read and write access is applied at this level.
When a user has [authenticated](#auth) with FlakeHub Cache, it determines which slices you're allowed to access and combines those slices into a single **view** of the cache.
With this view abstraction, FlakeHub Cache can make decisions like these:
* User `devops_aficionado_123` from the `WidgetsDotCom` org *may* pull from the cache for the `WidgetsDotCom/devops` flake.
* User `AnaBooper` from the `WidgetsDotCom` org *may not* pull from `WidgetsDotCom/super-secure` flake.
Although these users are in the same organization, WidgetsDotCom can make access decisions on a per-flake basis and thus choose whichever access patterns and collaboration models they wish.
This model is certainly more robust from a security standpoint but it also:
1. **Is much simpler**.
When using Nix, you only need to configure FlakeHub Cache; there's no need to distribute public keys for a multitude of them.
1. **Offers better performance**.
Using only one platform for caching means fewer network round trips and fewer authentication handshakes.
In addition, we run FlakeHub Cache in a variety of regions with CDN-backed storage.
This results in a clear speed-up in your Nix builds.
In addition to access control, things like garbage collection are also configurable at the flake level.
## A new caching model \{#model}
Two core aspects of using FlakeHub Cache make it a significant departure from other available cache systems:
1. **No public caches**.
That's right: FlakeHub Cache doesn't allow you to expose public caches like [cache.nixos.org][cno].
You can pull from the cache for a specific flake only if you're (a) authenticated and (b) have [permission](#access-control).
That means that you can't even *accidentally* make caches for specific flakes truly public.
We're open to exploring public caches in the future but would only do so with due care.
1. **Push only from CI**.
At the moment, when you create FlakeHub Cache authentication tokens, those only apply to *pulling* from the cache.
You can't *push* to the cache in an ad-hoc way using a CLI or other tool; you can only push from [GitHub Actions][actions], particularly the [Magic Nix Cache Action][magic-action] (with [GitLab support coming soon][gitlab]).
If you enroll in the beta, you can enable pushing with this one-liner in your Actions configuration:
```yaml
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v3
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix flake check
```
With that in place, anything that you `nix build` is automatically pushed to the cache—if authorized to!—without any need for separate push commands.
## A bold step forward
We firmly believe that FlakeHub Cache is the first enterprise-ready cache in the Nix ecosystem.
Public caches and caches without granular access control are not suited for a broad range of use cases and even whole industries, particularly those subject to exacting compliance standards, regulatory regimes, and security requirements.
If you're compelled by the power of Nix but have concerns about existing solutions, this might be precisely the sea change you've been waiting for.
On top of that, Determinate Systems is the only [SOC2]-compliant vendor in the Nix ecosystem, which should make otherwise-difficult discussions with the higher-ups go much more smoothly.
You can try out FlakeHub Cache for yourself soon.
Sign up below, configure one of your GitHub Actions runs to use the Magic Nix Cache Action, make a small update to your Nix configuration, and you'll experience this sea change first hand.
[actions]: https://docs.github.com/en/actions
[caching]: https://zero-to-nix.com/concepts/caching
[cno]: http://cache.nixos.org
[derivations]: https://zero-to-nix.com/concepts/derivations
[detsys]: /
[devenv]: https://zero-to-nix.com/concepts/dev-env
[flakehub]: https://flakehub.com
[flakes]: https://zero-to-nix.com/concepts/flakes
[github]: https://github.com
[gitlab]: https://gitlab.com
[jwt]: https://jwt.io
[magic-action]: https://github.com/DeterminateSystems/magic-nix-cache-action
[nix]: https://zero-to-nix.com
[nix-serve]: https://nixos.org/manual/nix/stable/package-management/binary-cache-substituter
[packages]: https://zero-to-nix.com/concepts/packages
[saml]: https://en.wikipedia.org/wiki/SAML_2.0
[soc2]: https://www.imperva.com/learn/data-security/soc-2-compliance/
[sso]: https://en.wikipedia.org/wiki/Single_sign-on
[store]: https://zero-to-nix.com/concepts/nix-store
---
**KVM on GitHub Actions**
Published: November 29, 2023
URL: https://determinate.systems/blog/kvm-on-github-actions
[KVM] is the most widely used virtualization framework for Linux due to its close integration with the Linux kernel.
It's become a mainstay in the [Nix] community because NixOS's innovative [test framework][tests] relies on it for virtualization.
I'm pleased to announce that last week the [Determinate Nix Installer Action][action] began enabling KVM on Linux [GitHub Actions runners][actions] by default.
Add one line of code to your YAML Actions configuration and you're good to go:
```yaml
- uses: DeterminateSystems/determinate-nix-action@v3
```
## What has changed
Despite KVM's popularity, Nix folks have been essentially blocked from using KVM in [GitHub Actions][actions].
This is a shame for many reason, but above all because KVM is [required] for running [NixOS tests][tests].
This means that a major piece of Nix and NixOS has required solutions like custom runners—which is fine but often a pretty heavy lift.
And beyond NixOS tests, this also unblocks fun VM-related things like running [Firecracker] VMs in Actions.
## Requirements
Please note that GitHub's policies stipulate that KVM is only available on larger, paid Actions runners.
Our testing reveals, however, that they generously provide it to public projects as well.
## How we did it
Well, it turns out that Linux GitHub Actions runners _do_ support KVM.
They just need a little coaxing (in the form of a few setup commands).
You can see how we do it in [this pull request][pr].
## Disabling KVM
As I mentioned above, the baseline GitHub Actions configuration for the Determinate Nix Installer will automatically enable KVM:
```yaml
- uses: DeterminateSystems/determinate-nix-action@v3
```
You can, however, disable this behavior if you need to:
```yaml
- uses: DeterminateSystems/determinate-nix-action@v3
with:
kvm: false
```
## Conclusion
This is indeed a small change but one that I believe could have a major impact in the Nix community.
Thorough testing is crucial for using [NixOS] in production environments and I'm relieved to see this barrier to entry removed.
If you have other ideas for how we can improve the experience of using Nix on GitHub Actions—or more broadly—please [get in touch][support] and let us know how!
[action]: https://github.com/DeterminateSystems/determinate-nix-action
[actions]: https://docs.github.com/actions
[firecracker]: https://firecracker-microvm.github.io/
[kvm]: https://linux-kvm.org/page/Main_Page
[nix]: https://zero-to-nix.com/concepts/nix
[nixos]: https://zero-to-nix.com/concepts/nixos
[pr]: https://github.com/DeterminateSystems/determinate-nix-action/pull/56
[required]: https://github.com/NixOS/nixpkgs/blob/bd9c192fc0715fce7b28d33c41f854c5219c2de8/nixos/lib/testing/run.nix#L44
[support]: mailto:support@flakehub.com
[tests]: https://hydra.nixos.org/project/nixos
---
**A graphical installer for Nix**
Published: November 24, 2023
URL: https://determinate.systems/blog/graphical-nix-installer
> Update, 2024-11-08: Determinate Nix's graphical installer has replaced this post's installer.
> The original graphical installer discussed here is no longer maintained, and the links now point to the Determinate Nix package.
Here at Determinate Systems, our core goal is to [make Nix better][better] along as many axes as we possibly can.
That has meant building major platforms geared toward Nix users, like [FlakeHub].
But sometimes smaller interventions can make a big difference and today I'm eager to tell you about one: a **graphical installer** for Nix users on macOS.
As Nix folks—and especially the Nix curious—know all too well, getting Nix to work on a machine in the first place is _not_ trivial.
We built the [Determinate Nix Installer][dni] to make this as painless as possible on Linux, macOS, and Windows Subsystem for Linux (WSL).
Invoke just one command and you're off to the races with a next-generation installer written in [Rust]:
```shell
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
```
But this week we decided to take things even further and offer a **one-click install** option for macOS.
Give it a try right now:
Don't worry: if you already have Nix installed, this won't harm your system, as we've built the Determinate Nix Installer to gracefully handle previous Nix installs.
## Why a graphical installer?
This is a great question because the standard Determinate Nix Installer requires just a single command.
Why not stop there?
Two reasons:
1. The package is [signed] by Determinate Systems and checked for malware.
This means that the chain of cryptographically validated trust is extended further than before and that you can be confident that the installer arrived intact (as in: not tampered with).
We have more work to do here but this is a great first step.
1. The package is readily usable by [**mobile device management**][mdm] (MDM) platforms like [Mosyle].
This can serve as a powerful way to distribute Nix to users in managed environments—without needing to run a shell command themselves.
## Future plans
Nix users on the Determinate Systems [Discord server][discord] have responded quite positively to the graphical installer thus far.
But we're not done just yet.
We're currently working on two things:
1. Providing ability to customize the installation.
Unfortunately, you can't provide custom parameters for a given `.pkg` installer, which means you can't tweak the knobs of your Nix installation _during_ the installation process itself.
But what you _can_ do is generate custom `.pkg` files, and we're currently working on ways to take user preferences and generate them on the fly.
1. Expanding the range of what the Determinate Nix Installer—via the graphical installer—can achieve.
That means going beyond installing Nix to installing Nix-related tools like [Home Manager][hm] and [nix-darwin].
Imagine clicking a button and getting a whole suite of Nix tools ready to go—or clicking nothing and letting an MDM platform do the work.
If you're on macOS, give the graphical installer a try and let us know how it goes on [Discord].
We look forward to hearing from you.
[better]: /blog/we-want-to-make-nix-better
[discord]: https://discord.gg/invite/a4EcQQ8STr
[dni]: https://github.com/DeterminateSystems/nix-installer
[flakehub]: https://flakehub.com
[hm]: https://github.com/nix-community/home-manager
[mdm]: https://en.wikipedia.org/wiki/Mobile_device_management
[mosyle]: https://mosyle.com
[nix-darwin]: https://github.com/LnL7/nix-darwin
[rust]: https://rust-lang.org
[signed]: https://developer.apple.com/developer-id
---
**Lessons from 1 million Nix Installs**
Published: November 7, 2023
URL: https://determinate.systems/blog/lessons-from-1-million-nix-installs
Nix has an [official installer][installer] that I feel has served the community reasonably well over the years.
From the beginning it had some issues that made me long for a better alternative—but not so much that I considered rethinking the installer from the ground up.
That changed about a year ago.
I was struggling with [the installer's Bash scripts][tmpdir] and trying to handle an edge case around temporary directories.
Fixing this bug took _nearly a week_, and the end result?
_Five_ lines of code—and even when we were done we still weren't sure if it was right.
At that moment, I knew that we at Determinate Systems needed to start with a clean slate and build something great, not in Bash but rather in the much more robust and expressive [Rust].
From the beginning we focused on reliability, user experience, and providing a modern Nix experience out of the box.
And I firmly believe that we have succeeded.
The [Determinate Nix Installer][dni], as we came to call it:
- **enables flakes by default.**
Nix users both recent and long term have overwhelmingly adopted [flakes].
Our installer always enables flakes and offers a stability guarantee that any flakes that work today will work tomorrow.
Users of the Determinate Nix Installer don't need to hesitate or worry about adopting flakes.
- **has an uninstaller.**
Don't like Nix?
No worries.
Our installer has a single-command, safe, and thorough uninstaller.
- **works almost anywhere.**
Install to Linux, macOS, or WSL, on a Steam Deck, inside a Docker container, on an OSTree distro, you name it.
If it doesn't work, it's a bug!
- **has a [GitHub Action][action].**
Add `uses: DeterminateSystems/determinate-nix-action@v3` to your GitHub Actions workflow and you're done.
You'll get a safe and stable installation of Nix, with flakes enabled, and all the predictability you're looking for in CI.
- **survives macOS upgrades.**
This might seem a bit funny-sheesh, but it is true.
We [wrote a whole post][posts-nix-survival-mode-on-macos] on it—so check it out.
- **isn't scary.**
Many folks over the years have uninstalled Nix because the installer was scary.
The Determinate Nix Installer is different.
Its output is succinct, clear, and feels safe.
A user's first introduction is important and we leave the user feeling good about installing Nix.
So just _how_ successful has the Determinate Nix Installer been?
Today I'm pleased to announce that it has successfully installed Nix **over one million times** since we first [introduced it in January of 2022][posts-determinate-nix-installer].
## Targeting 100% success
We strive for a 100% success rate when installing Nix.
That doesn't mean that we put the files in the right place and call it good.
Nix has to _work_ the way that users expect it to.
A "successful failure" is still a failure in our eyes.
We're not big fans of software "phoning home."
Nobody loves it, and every change we make to our diagnostics receives careful reviews and strong critique on principle.
At the same time, we could not make the most reliable installer without it.
The Determinate Nix Installer collects a minimal amount of diagnostic data after every installation.
This data includes the OS and architecture of the computer, whether the install succeeded, and some sanitized amount of failure information, such which part of the installation process failed.
Collecting this data is a critical component of improving the installer and targeting the most important problems that users are facing.
Reaching 100% may not be possible but we have the results to prove we're striving to give users the best experience every time.
Overall, we're tracking approximately a **99.4%** success rate.
## Rolling deployments
One important way we're able to retain our success rate is through our carefully orchestrated, rolling deployments.
We don't just flip a switch and move 100% of our users to new releases all at once.
This is risky and it doesn't treat users with the consideration they deserve.
Nix and the Determinate Nix Installer are load-bearing components of our users' stacks and we have to respect that.
Our releases start by rolling out to only 20% of requests from GitHub Actions.
We start with GitHub Actions because the environment is ephemeral and failure cases can be resolved by restarting the job.
This means that users on long-term devices don't get a bad experience and CI users are likely to hit "re-run" when they encounter a weird edge case.
We carefully track the new release and monitor to see if users are experiencing an increase in installation failures.
But we don't stop there.
Like I mentioned earlier, our goal is not just to throw Nix down on the host and call it good.
Our goal is to deliver a working version of Nix that doesn't break users' setups.
To accomplish this, [our GitHub Action][action] reports back anonymized summary data for public GitHub Actions workflows.
The data is a little bit noisy but it is also valuable, and in practice we find that the rate of workflow failures is consistent between two releases unless Nix or the installer is broken.
This data along with some diagnostics data enables us to identify problems and regressions in Nix itself for real users in a way that nobody else in the Nix landscape is doing.
Over time, we carefully ramp up the GitHub Actions installations until we reach 100%, and alongside that we also ramp up the upgrade for users outside of CI.
Because our failure rate in CI is so low, we're able to take careful, measured steps to roll out new features and updates without big-bang releases that can spoil many an afternoon.
The long tail of error conditions is long but I do believe that our results speak for themselves.
## Lessons learned on our way to 100%
User machines in particular have a uniquely _fascinating_ history (to put it diplomatically).
### macOS' security model is robust
Installing Nix on macOS means reckoning with an ever-tightening security model.
This is generally good for users but it means that Nix and its installer have to constantly keep up.
Our installer is written in Rust, which means that adapting to the continuous upstream changes is safer; we're not fighting the language as the official installer must do with Bash.
Our installer (and uninstaller!) successfully navigates configuring `synthetic.conf` and `fstab` and also creating APFS volumes—with encryption, and more.
Using Rust has been crucial to doing things the right way.
We're also able to do experiments and make improvements like switching from named APFS volumes to using UUIDs, which enables us to solve tricky problems surrounding [systems that boot without the Nix store mounted][nix-installer-issue-212].
### MUSL builds and nscd/sssd on Linux
For portability across Linux distributions, the Determinate Nix Installer is statically compiled using [MUSL].
In Rust, this means targeting `x86_64-unknown-linux-musl` and the `crt-static` target feature as documented in the [Rust Reference][rust-static-dynamic-runtime].
This unfortunately brings a new set of unique issues.
During one step of the installation process we use [`nix::unistd::User`][`nix::unistd::User`], which uses the [`getpwnam_r`][`libc::getpwnam_r`] syscall.
We've received [several reports][nix-installer-issues-nscd] that indicate that programs like `nscd` and `sssd` can override the `getpwnam_r` syscall.
In these situations, one workaround is to build our installer yourself with `cargo install nix-installer` and run that.
But we don't love this solution and we hope to provide something more compelling, such as creating [glibc]-based release binaries.
### Creating and deleting users
Serially creating ~32 users for Nix takes an annoyingly long time, measuring _seconds_ per user on some machines.
Early in development we experimented with creating users in parallel to speed up the process.
This turned out to be problematic on Mac and Linux due to locking and other parallelism-related issues.
We also looked at directly editing `/etc/passwd` and other files, but we are concerned this may cause further issues in enterprise environments with central user directories.
In addition, we adopted the [`auto-allocate-uids`][`auto-allocate-uids`] feature from Nix, which did make installation much faster but caused other issues.
On macOS, for example, we experienced problems building Nix (of all things) because `whoami` no longer worked.
We had problems on Linux, too.
In [issue #539][nix-installer-issue-539] we noticed that some distributions experienced errors like `setting uid: invalid argument`.
We ultimately rolled the feature back but one day we'd love to find a solution that would let us adopt it again.
And the fun continues.
In [issue #33][nix-installer-issue-33] we found that deleting users on certain Macs sometimes ends with a permissions error.
After quite a bit of investigation, as well as referencing articles like [Can't delete a macOS user with dscl][cant-delete-a-macos-user-with-dscl-resolution] and [When you "can't" delete a user in MacOS][when-you-cant-delete-a-user-in-macos], we uncovered the issue.
It seems that you can't delete users on macOS if nobody has logged into the machine graphically.
This was a big problem for us since we [run a macOS build farm][mac-build-farm] dedicated to building and testing the installer.
We still don't have an automatic fix but we do detect the error and provide instructions on how to resolve it.
### Nix's SSL certificate story needed improvement
Issues like [#289][nix-installer-issue-289] and later [#516][nix-installer-issue-516] made it evident that the existing `NIX_SSL_CERT_FILE` environment variable was causing some problems for certain installations, as well as confusion in some users.
Running `nix build`, for example, would sometimes produce errors like this:
```
warning: error: unable to download '...': SSL peer certificate or SSH remote key was not OK (60); retrying in 337 ms
```
The problem appeared to be stem from inconsistencies in how `NIX_SSL_CERT_FILE` was being handled.
During discussions with Eelco we concluded that the best solution would be to lift the `NIX_SSL_CERT_FILE` into a configuration option inside users' `nix.conf` [nixos/nix#8062][nix-pr-8062] configuration files.
This appears to have solved most of the issues we were seeing.
### Uninstallation order is important
A recurring issue that cropped up on our issue boards was a positively _bizarre_ CA certificate issue on Macs characterized by [pull request #608][nix-installer-pr-608].
Our first few reports made little sense.
Why was Nix trying to access `/etc/ssl/certs/ca-certificates.crt`?
That path doesn't normally exist on Mac and the install process doesn't involve it!
[Reproducing the issue][nix-installer-pr-528-comment-1629394069] required these steps:
1. Install Nix
2. Install [`nix-darwin`][nix-darwin]
3. Uninstall Nix either with `/nix/nix-installer uninstall` or the [official guide][nix-reference-uninstall]
4. Reinstall Nix
Uninstalling Nix before uninstalling `nix-darwin` leaves a Launch Daemon called `org.nixos.activate-system`.
Leaving this Launch Daemon lingering causes issues with the `NIX_SSL_CERT_FILE` environment variable, which in turn spoils reinstalls.
This issue prompted us to add new pre-install and pre-uninstall checks to warn about the issue before you hit it.
There is a [workaround][nix-installer-pr-608] and we hope that a future release of the installer will provide a robust cure for this issue.
### Containers are complicated
There are several popular container runtimes that differ in subtle ways.
Installation is pretty normal when targeting a [Podman] container with [systemd], for example, but Docker containers can't run systemd, which complicates the installation.
In some runtimes, Nix's sandboxing isn't a viable option due to highly restrictive sandboxing of the container itself.
At one point, we made a matrix of different options that worked for Podman and Docker but the complexity got the best of us.
In the end, we found [two configurations][nix-installer-readme-containers] that worked in the most common use cases.
It feels like there is still A story to be told on this particular issue and we'd be glad to find a better solution.
## Thank you to our collaborators
Whether it's a carefully described issue, a drive-by pull request, or even seeing your friendly faces at the [Installer Working Group][nix-discourse-installer-wg] meetings, we want to say **thank you** 🎉😊 for collaborating with us on this project.
It's been extremely uplifting to be able to participate in these greater community discussions.
We continue to hope that the upstream project adopts the Determinate Nix Installer for itself.
In particular, a big thank you to [Abathur][nix-discourse-user-abathur] and [Mkenigs][nix-discourse-user-mkenigs] for continued in-depth collaboration.
## What's next?
An early design choice was that our installer should have a public API for building custom installers on top of it.
Ultimately, this hasn't received a lot of interest and has made creating the user experience we want much more complicated.
We haven't yet decided if we want to keep this API, but if this is important to you, please let us know on our [Discord][discord-invite].
We're well on our way to our next million installations, but before we get there it'd be great to call it 1.0.0.
If you'd like to chat about Nix and get help with flakes, please join us on our flake-forward [Discord][discord-invite]!
[action]: https://github.com/DeterminateSystems/determinate-nix-action
[cant-delete-a-macos-user-with-dscl-resolution]: https://web.archive.org/web/20240422065844/https://it.megocollector.com/macos/cant-delete-a-macos-user-with-dscl-resolution/
[discord-invite]: https://discord.gg/bU9enxBwJt
[dni]: https://github.com/DeterminateSystems/nix-installer
[flakes]: https://zero-to-nix.com/concepts/flakes
[installer]: https://nixos.org/download
[glibc]: https://www.gnu.org/software/libc
[mac-build-farm]: https://buildkite.com/blog/ephemeral-mac-os-builds-buildkite-nix-tailscale
[musl]: https://www.musl-libc.org
[nix-darwin]: https://github.com/LnL7/nix-darwin
[nix-discourse-installer-wg]: https://discourse.nixos.org/t/nix-installer-workgroup/21495
[nix-discourse-user-abathur]: https://discourse.nixos.org/u/abathur/summary
[nix-discourse-user-mkenigs]: https://discourse.nixos.org/u/mkenigs/summary
[nix-installer-issue-212]: https://github.com/DeterminateSystems/nix-installer/issues/212
[nix-installer-issue-33]: https://github.com/DeterminateSystems/nix-installer/issues/33
[nix-installer-issue-289]: https://github.com/DeterminateSystems/nix-installer/issues/289
[nix-installer-issue-516]: https://github.com/DeterminateSystems/nix-installer/issues/516
[nix-installer-issue-539]: https://github.com/DeterminateSystems/nix-installer/issues/539
[nix-installer-issues-nscd]: https://github.com/DeterminateSystems/nix-installer/issues?q=nscd
[nix-installer-pr-528-comment-1629394069]: https://github.com/DeterminateSystems/nix-installer/issues/528#issuecomment-1629394069
[nix-installer-pr-608]: https://github.com/DeterminateSystems/nix-installer/issues/608
[nix-installer-readme-containers]: https://github.com/DeterminateSystems/nix-installer#in-a-container
[nix-pr-8062]: https://github.com/NixOS/nix/pull/8062
[nix-reference-uninstall]: https://nixos.org/manual/nix/stable/installation/uninstall
[podman]: https://podman.io
[posts-nix-survival-mode-on-macos]: /blog/nix-survival-mode-on-macos
[posts-determinate-nix-installer]: /blog/determinate-nix-installer
[rust]: https://rust-lang.org
[rust-static-dynamic-runtime]: https://doc.rust-lang.org/stable/reference/linkage.html#static-and-dynamic-c-runtimes
[systemd]: https://systemd.io
[tmpdir]: https://github.com/NixOS/nix/pull/6916
[when-you-cant-delete-a-user-in-macos]: https://web.archive.org/web/20240422065842/https://www.aixperts.co.uk/?p=214
[`auto-allocate-uids`]: https://nixos.org/manual/nix/stable/contributing/experimental-features#xp-feature-auto-allocate-uids
[`libc::getpwnam_r`]: https://docs.rs/libc/latest/libc/fn.getpwnam_r.html
[`nix::unistd::User`]: https://docs.rs/nix/latest/nix/unistd/struct.User.html
---
**Nix Survival Mode: sheltering Nix from macOS upgrades**
Published: October 25, 2023
URL: https://determinate.systems/blog/nix-survival-mode-on-macos
Tens of thousands of macOS users install Nix using the [Determinate Nix Installer][determinate-nix-installer], and I'm one of them.
Last February, [we introduced the Determinate Nix Installer][dni-blog-post] to solve a multitude of reliability problems and failure conditions.
This has been a rousing success, but we always aspire to reduce friction and improve the user experience.
Our goal is 100% success with zero fuss.
Despite a rounding error of install-time failures, hilariously, most Nix installations are broken after macOS upgrades.
Not great: this is not the user experience we're going for!
People usually like software better when it stays installed, and you'd think Nix—striving for predictability—wouldn't do this.
Well, good news:
**Nix installations from the Determinate Nix Installer after version v0.14 survive macOS upgrades.** 🎉
The latest release of our installer now creates a global Launch Daemon which restores the necessary pieces of Nix's installation at boot time.
That means the Nix you installed stays installed, eliminating the largest source of friction for macOS users and demonstrating our commitment to being the most reliable, resilient, and safest way to install Nix.
Try it on your Mac today:
```shell
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
```
Or integrate the most reliable Nix installer in GitHub Actions:
```yaml
- uses: DeterminateSystems/determinate-nix-action@v3
```
[determinate-nix-installer]: https://github.com/DeterminateSystems/nix-installer
[dni-blog-post]: /blog/determinate-nix-installer
---
**FlakeHub updates: lightning-fast mirroring and webhook integrations**
Published: October 6, 2023
URL: https://determinate.systems/blog/flakehub-updates
We're a [little over a month][fh-post] into the life of [FlakeHub] and it's been quite the journey so far.
Well over 150 flakes have been published by almost 100 GitHub orgs and we've been hard at work bringing more and more features to the platform, including [a CLI tool called `fh`][fh] and [flake schemas][schemas].
In this post, I'd like to talk about some recent updates as well as some fun stuff on the way.
## Faster mirroring
We've gotten tons of wonderful feedback from enthusiastic FlakeHub users on [Discord] and other venues, including a handful of really compelling feature requests.
Some FlakeHub users have requested more timely updates to widely used flakes like [Nixpkgs] and we're happy to say that we now update all flakes **within 60 seconds of an upstream release**.
_[Mirrored]_ flakes are available on FlakeHub but haven't yet been published by the original authors.
Mirroring enabled us at Determinate Systems to provide widely used flakes to users upon launch.
Initially, we published Nixpkgs twice a day, which meant that people relying on it could see up to a 12-hour lag between update and delivery.
But user feedback has convinced us that this isn't nearly frequent enough, so we've implemented a long-running process that listens for pushes to Nixpkgs and immediately publishes them to FlakeHub.
While we do want upstream projects to publish directly to FlakeHub, we hope that this will tide over FlakeHub users (and us at DetSys!) in the meantime.
## Beta webhooks and integrations
FlakeHub users can now **subscribe to webhooks for any published flake**.
Users have been asking for pub/sub notifications for updates to projects like Nixpkgs, and FlakeHub is now poised to deliver.
Some use cases for this include triggering [`flake.lock` updates][updates] or an internal rebase for a fork.
This feature is currently in **limited beta**, so contact [support@flakehub.com][support] if you'd like to be an early tester.
[Keep an eye out][blog], we have more on its way!
[blog]: /
[discord]: https://determinate.systems/discord
[fh]: /blog/flakehub-cli
[fh-post]: /blog/introducing-flakehub
[flakehub]: https://flakehub.com
[mirrored]: https://docs.determinate.systems/flakehub/concepts/mirroring
[nixpkgs]: https://github.com/NixOS/nixpkgs
[schemas]: /blog/flake-schemas
[support]: mailto:support@flakehub.com
[updates]: https://github.com/marketplace/actions/update-nix-flake-lock
---
**Creating and modifying flakes using the FlakeHub CLI**
Published: September 20, 2023
URL: https://determinate.systems/blog/fh-updates
Last month we at [Determinate Systems](/) announced the [initial release][flakehub-post] of [FlakeHub], a brand new platform for publishing and exploring [Nix flakes][flakes], with features like [SemVer] for flakes and robust [search].
Soon thereafter, we released [`fh`][fh], a CLI tool for interacting with FlakeHub.
The initial release of `fh` offered a modest set of capabilities, including the ability to [list][fh-list] and [search][fh-search] all publicly listed flakes.
But recently we've been iterating on the tool in earnest and today we'd like to announce two new features: [`fh add`](#fh-add) and [`fh init`](#fh-init), both of which substantially improve the ergonomics of working with [`flake.nix` files][flakes].
Try out `fh` right now:
```shell
nix run "https://flakehub.com/f/DeterminateSystems/fh/0.1.5.tar.gz"
```
See the [README instructions][install] to install it on your system.
## `fh add`
The [`fh add`][fh-add] command enables you to quickly add new [flake inputs][inputs] to an existing flake.
This command, for example, would add [Nixpkgs] to your `flake.nix`:
```shell
fh add NixOS/nixpkgs
```
If you specify a flake reference as `:org/:project`, as in this example, `fh` infers that you mean a [FlakeHub] reference and searches to ensure the flake exists.
It then adds a reference to the latest version of the flake.
The resulting `inputs` block may look something like this (notice the FlakeHub address and the [specific version][flakehub-versions]):
```nix
{
inputs = {
nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.490944.tar.gz";
# Other inputs
};
}
```
You can also use `fh add` to add non-FlakeHub references:
```shell
fh add github:DeterminateSystems/nuenv
```
## `fh init`
Though it's nice to have a convenient way to update [flakes] that already work, creating new flakes from scratch can be a bit of a chore.
I've done it enough times that it's now ingrained in my muscle memory but I don't look forward to it and we at Determinate Systems suspect that _no one_ does.
So we added an [`fh init`][fh-init] command to the FlakeHub CLI.
All you have to do is run it in the root of a project and 🪄🦄🧚♀️ magical things happen 🪄🦄🧚♀️:
```shell
cd /path/to/your/project
fh init
```
`fh init` assumes that you don't yet have a `flake.nix` at the root of your project (and if you do, it only overwrites it if you explicitly opt for that).
Then it looks at the contents of your project and asks you a series of questions about what you want.
At the end of that process, it outputs a `flake.nix` file that you can immediately use in your project.
That's it!
Try it in a flake-less project now:
```shell
nix run "https://flakehub.com/f/DeterminateSystems/fh/0.1.5.tar.gz" -- init
```
Here's an example `fh init` flow:
- If your project has a `Cargo.toml` file, it asks if this is indeed a [Rust] project.
- If you say yes, then it checks for a [`rust-toolchain`][rust-toolchain] or [`rust-toolchain.toml`][rust-toolchain] file.
- If one is present, it uses that to create your Rust environment; if not, it uses the latest stable version of Rust.
[Similar flows][handlers] are currently available for Go, Java, JavaScript, PHP, Python, Ruby, and Zig.
Beyond these language-specific helpers, `fh init` also enables you to:
- Provide a description of the flake
- Include helpful doc comments explaining various aspects of the flake
- Include common utilities like [curl], [jq], [Git], and [nixpkgs-fmt] (the Nix formatter that we prefer at Determinate Systems)
- Generate a flake-friendly [direnv] configuration file
- Select which systems you want the flake to support (`x86_64-linux`, `aarch64-darwin`, etc.)
- Provide custom environment variables
As we usually do at DeterminateSystems, we have strong—though hopefully informed—opinions about how things should be, and flakes are, [unsurprisingly][stable-flakes], no exception to that.
Some opinions that we baked into `fh init`:
- Rather than using libraries like [`flake-utils`][flake-utils] or [`flake-parts`][flake-parts], it generates plain old Nix functions as helpers for generating [per-system outputs][system-specificity].
- It always adds [Nixpkgs] as an input, as it's necessary for pretty much all the flakes that `fh init` generates.
It does, however, enable you to select which version of Nixpkgs you want.
But `fh init` is just a starter.
You're always free to customize at will; after all, it's just Nix code!
We should also note here that `fh init` is different from [flake templates][templates] and the related [`nix flake init`][nix-flake-init] command.
Flake templates essentially copy pre-existing files—including `flake.nix` files if you want—into your current directory (or a new directory).
`fh init`, conversely, takes the specifics of your project and your desires into account.
## More on the way for `fh` \{#more}
We're confident that `fh add` and `fh init` will polish off some of the rough edges of working with flakes.
But we have plans to improve `fh init` pretty dramatically in the near future, including:
- Support for generating [flake outputs][outputs] beyond [`devShells`][dev-env], such as package outputs
- More language- and tool-specific interactive flows
In the meantime, please don't hesitate to provide feedback in the form of [issues] and [pull requests][prs].
Or join us on [Discord] if you just want to chat about [FlakeHub].
[curl]: https://curl.se
[dev-env]: https://zero-to-nix.com/concepts/dev-env
[direnv]: https://direnv.net
[discord]: https://discord.gg/invite/a4EcQQ8STr
[fh]: /blog/flakehub-cli
[fh-add]: https://github.com/determinateSystems/fh#add-a-flake-published-to-flakehub-to-your-flakenix
[fh-init]: https://github.com/DeterminateSystems/fh#initialize-a-new-flakenix-from-scratch
[fh-list]: https://github.com/DeterminateSystems/fh#listing-flakes-organizations-and-versions
[fh-search]: https://github.com/DeterminateSystems/fh#searching-published-flakes
[flake-parts]: https://github.com/hercules-ci/flake-parts
[flake-utils]: https://flakehub.com/flake/numtide/flake-utils
[flakehub]: https://flakehub.com
[flakehub-post]: /blog/introducing-flakehub
[flakehub-versions]: https://docs.determinate.systems/flakehub/concepts/semver
[flakes]: https://zero-to-nix.com/concepts/flakes
[git]: https://git-scm.com
[handlers]: https://github.com/DeterminateSystems/fh/tree/main/src/cli/cmd/init/handlers
[inputs]: https://zero-to-nix.com/concepts/flakes/#inputs
[install]: https://github.com/DeterminateSystems/fh#installation
[issues]: https://github.com/determinateSystems/fh/issues
[jq]: https://jqlang.github.io/jq
[nix-flake-init]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-init
[nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs
[nixpkgs-fmt]: https://github.com/nix-community/nixpkgs-fmt
[outputs]: https://zero-to-nix.com/concepts/flakes/#outputs
[prs]: https://github.com/determinateSystems/fh/pulls
[rust]: https://rust-lang.org
[rust-toolchain]: https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file
[search]: https://flakehub.com/search?q=nix
[semver]: https://docs.determinate.systems/flakehub/concepts/semver
[stable-flakes]: https://discourse.nixos.org/t/experimental-does-not-mean-unstable-detsyss-perspective-on-nix-flakes/32703
[system-specificity]: https://zero-to-nix.com/concepts/system-specificity
[templates]: https://zero-to-nix.com/concepts/flakes/#templates
---
**Experimental does not mean unstable**
Published: September 6, 2023
URL: https://determinate.systems/blog/experimental-does-not-mean-unstable
Although Nix flakes are currently marked as experimental, the on-the-ground experience of actually _using_ flakes has been quite stable since their initial release in November 2021.
Even the so-called "unified CLI," also marked as experimental, has seen remarkably few breaking changes in that time.
Evidence from GitHub strongly suggests that the Nix community is moving in the direction of flakes and the unified CLI _en-masse_.
And I hear from more recent Nix adopters over and over again that flakes were crucial to their learning journey.
I believe that flakes are stable, and I'm not alone.
My position is that the experimental flag should have been removed ages ago—or never introduced in the first place.
I don't think that flakes need a lengthy stabilization process.
Nix users of all stripes have been using flakes in production for years and to great effect.
The "experimental" label needs to be removed.
Flakes are a proven technology and deserve to be officially recognized as such.
Nonetheless, flakes in their current form have detractors, and those detractors make some reasonable points.
I want to address three common criticisms of flakes.
1. **Source explosion**.
Many of you have probably seen the article about [1,000 instances of Nixpkgs](https://zimbatm.com/notes/1000-instances-of-nixpkgs).
It's a good read but I disagree with the conclusions.
With evaluation caching and the functional nature of Nixpkgs, source explosion is generally mitigated using the `follows` mechanism built into flakes, which reduces the number of unique Nixpkgs instances involved in an evaluation.
And by embedding further support for version boundaries that [FlakeHub] introduces, the Nix locking mechanism can even make reasonable dependency unification choices automatically.
2. **Cross-compilation**.
Some say that cross-compilation using flakes isn't great.
Flakes and the unified CLI indeed don't offer specific support for cross-compilation—but the old Nix CLI and channels didn't offer direct support either.
Improving cross-compilation in Nix is a large though highly worthy project.
But it has nothing to do with flakes and thus no bearing on stabilization.
3. **Lockfile format**.
I've seen discussions around this and the terms are always vague.
Sometimes, lockfiles can become rather large.
I don't have a lot of sympathy for this problem, when Nixpkgs' git history is gigantic and computing is relatively cheap.
Nonetheless, this problem is also being worked on and does not need to be solved prior to stabilization, as a new mechanism can increment the `flake.lock` version field.
I think that #1 is a fair criticism and an area where I'd like to see forward progress.
But I do _not_ see it as a compelling reason to continue marking flakes as experimental.
We can declare flakes as stable **today** and find ways to address source explosion under the banner of flakes as a stable feature of Nix.
Could flakes be improved?
Yes.
I do acknowledge that there are some fundamental issues that should be addressed—this is true of just about any technology.
But the "experimental" flag should not, in my view, be interpreted as "unstable."
It would be reckless to break backwards compatibility in how flakes are evaluated, especially locked flakes.
Any changes to flakes should be made with due care and respect for the sake of the thousands of users incorporating Nix into their daily workflows.
Determinate Systems is committed to maintaining that compatibility in partnership and in collaboration with the Nix maintenance team.
We shouldn't let yet another NixCon, yet another year go by having only a vague sense of what it'll take to get flakes stabilized.
It's time to call good enough good enough.
I'm calling on the Nix team to remove the experimental flag by the end of the year and to thereby open a new chapter in Nix's history and pave the way for other worthy goals.
[flakehub]: https://flakehub.com
---
**`fh`: the CLI for FlakeHub**
Published: September 1, 2023
URL: https://determinate.systems/blog/flakehub-cli
Last week, we released [FlakeHub], a new platform for publishing, discovering, and using [Nix flakes][flakes].
We've been extremely pleased with the initial response: as of this post, more than [60 organizations][fh-orgs] have published a total of over [110 flakes][fh-flakes] to FlakeHub, and many Nix users have already incorporated the platform into their day-to-day workflows.
Although we're happy with the web UI for FlakeHub, we know that many workflows will require more programmatic access to the platform.
And so today we're announcing the initial release of [`fh`][fh], the CLI for FlakeHub.
To give an analogy, `fh` serves the same role for FlakeHub that the [`gh`][gh] CLI serves for GitHub.
## Getting started
First off, you'll need Nix 2.17 or above to use `fh`.
Check out the [FlakeHub docs][upgrade] for information on how to upgrade.
Once you have a compatible version of Nix, there are several ways to get started.
You can start a shell session with it installed using [`nix shell`][nix-env-shell]:
```shell
nix shell "https://api.flakehub.com/f/DeterminateSystems/fh/*.tar.gz"
```
You can also add it to a Nix [development environment][env] as in this `flake.nix`:
```nix
{
inputs = {
nixpkgs.url = "https://api.flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz";
fh.url = "https://api.flakehub.com/f/DeterminateSystems/fh/0.1.*.tar.gz";
};
outputs = { self, nixpkgs, fh, .. } @ inputs:
let
supportedSystems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ];
forEachSupportedSystem = f: nixpkgs.lib.genAttrs supportedSystems (system: f {
inherit system;
pkgs = import nixpkgs { inherit system; };
});
in
{
devShells = forEachSupportedSystem ({ pkgs, system }: {
default = pkgs.mkShell {
packages = [
fh.packages.${system}.fh
];
};
});
};
}
```
To install it `fh` in your home environment, we recommend using [Home Manager][hm].
## Commands
### `fh add` \{#add}
`fh add` adds a FlakeHub dependency to a flake.
Let's say that you've defined a flake in `flake.nix` in the current directory:
```nix
{
inputs.nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz";
outputs = { self, nixpkgs }: {
# Outputs here
};
}
```
This command would add the [`ipetkov/crane`][crane] flake:
```shell
fh add ipetkov/crane
```
The resulting `flake.nix`:
```nix
{
inputs.crane.url = "https://flakehub.com/f/ipetkov/crane/0.13.1.tar.gz";
inputs.nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz";
outputs = { self, nixpkgs, crane }: {
# Outputs here
};
}
```
`fh add` queries FlakeHub to find the most recent version of that input (in this case [0.13.1][crane-latest] for `ipetkov/crane`).
You can also use `fh add` with non-FlakeHub flake references.
Some examples:
```shell
fh add github:NixOS/patchelf
fh
```
### `fh list` \{#list}
`fh list` has three subcommands that are useful for getting a general idea of what's on FlakeHub right now:
- `fh list flakes` lists all the publicly listed [flakes][fh-flakes] on FlakeHub
- `fh list orgs` lists all the publicly listed [organizations][fh-orgs] on FlakeHub
- `fh list releases ` lists all the releases of a flake on FlakeHub.
Try `fh list releases tailscale/tailscale`, for example.
### `fh search` \{#search}
`fh search` enables you to search FlakeHub's [Algolia] index of flakes using an arbitrary string query.
Here's an example:
```shell
fh search rust
```
That returns this table:
```
+---------------------------------------------------------------------------------+
| Flake FlakeHub URL |
+---------------------------------------------------------------------------------+
| astro/deadnix https://flakehub.com/flake/astro/deadnix |
| carlthome/ml-runtimes https://flakehub.com/flake/carlthome/ml-runtimes |
| ipetkov/crane https://flakehub.com/flake/ipetkov/crane |
| kamadorueda/alejandra https://flakehub.com/flake/kamadorueda/alejandra |
| nix-community/fenix https://flakehub.com/flake/nix-community/fenix |
| nix-community/lanzaboote https://flakehub.com/flake/nix-community/lanzaboote |
| nix-community/nix-init https://flakehub.com/flake/nix-community/nix-init |
| nix-community/nixpkgs-fmt https://flakehub.com/flake/nix-community/nixpkgs-fmt |
| nix-community/patsh https://flakehub.com/flake/nix-community/patsh |
| ryanccn/nyoom https://flakehub.com/flake/ryanccn/nyoom |
+---------------------------------------------------------------------------------+
```
Well, that query returns these results _today_, but this will surely change in the future as more flakes are added, included [Rust]-related flakes.
You can also run more complex queries, like these:
```shell
fh search "nixos modules"
fh search "flake python"
```
## Going forward
`fh` is a modest tool but it brings real improvements to the experience around [Nix flakes][flakes].
It gives you quick access to the steadily expanding constellation of flakes on [FlakeHub], including search, and to programmatically add flakes to your projects.
In the near term, we have plans to support interactively creating new `flake.nix` files and to expand `fh`'s ability to modify existing flakes.
So keep up to date with the [`fh` flake][fh-flake] for future improvements.
[algolia]: https://algolia.com
[crane]: https://flakehub.com/flake/ipetkov/crane
[crane-latest]: https://flakehub.com/flake/ipetkov/crane/0.13.1
[env]: https://zero-to-nix.com/concepts/dev-env
[fh]: https://github.com/DeterminateSystems/fh
[fh-flake]: https://flakehub.com/DeterminateSystems/fh
[fh-flakes]: https://flakehub.com/flakes
[fh-orgs]: https://flakehub.com/orgs
[flakes]: https://zero-to-nix.com/concepts/flakes
[flakehub]: https://flakehub.com
[gh]: https://cli.github.com
[hm]: https://nix-community.github.io/home-manager
[nix-env-shell]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-env-shell
[rust]: https://rust-lang.org
[upgrade]: https://docs.determinate.systems/determinate-nix#determinate-nixd-upgrade
---
**Flake schemas: making flake outputs extensible**
Published: August 31, 2023
URL: https://determinate.systems/blog/flake-schemas
[Flakes] are a generic way to package [Nix] artifacts.
Flake output attributes are arbitrary Nix values, so they can be [packages], [NixOS modules][modules], [CI jobs][hydra], and so on.
While there are a number of "well-known" flake output types that are recognized by tools like the [`nix` CLI][cli]—[`nix develop`][nix-develop], for example, operates on the `devShells` output—nothing prevents you from defining your own flake output types.
Unfortunately, such "non-standard" [flake output][output] types have a big problem: tools like [`nix flake show`][nix-flake-show] and [`nix flake check`][nix-flake-check] don't know anything about them, so they can't display or check anything about those outputs.
The [`nixpkgs`][nixpkgs] flake, for instance, has a `lib` output that Nix knows nothing about:
```shell
# nix flake show nixpkgs
github:NixOS/nixpkgs/4ecab3273592f27479a583fb6d975d4aba3486fe
├───...
└───lib: unknown
```
This was a problem when we were creating [FlakeHub]: the FlakeHub web interface should be able to display the contents of a flake, including documentation, but we want to do so in an extensible way.
Today we're proposing a solution to this problem: **flake schemas**.
Flake schemas enable flakes to declare functions that enumerate and check the contents of their outputs.
Schemas themselves are defined as a flake output named `schemas`.
Tools like `nix flake check` and FlakeHub can then use these schemas to display or check the contents of flakes in a generic way.
In this approach, flakes carry their own schema definitions, so you are not dependent on some central registry of schema definitions—_you_ define what your flake outputs are supposed to look like.
Here is an example of what the outputs of a flake, extracted using that flake's schemas, look like in FlakeHub:
## Using flake schemas
While you can define your own schema definition (see below), usually you would use schema definitions provided by others.
We provide a repository named [`flake-schemas`][repo] with schemas for the most widely used flake outputs (the ones for which Nix has built-in support).
Declaring what schemas to use is straightforward: you just define a `schemas` output.
```nix
{
# `flake-schemas` is a flake that provides schemas for commonly used flake outputs,
# like `packages` and `devShells`.
inputs.flake-schemas.url = github:DeterminateSystems/flake-schemas;
# Another flake that provides schemas.
inputs.other-schemas.url = ...;
outputs = { self, flake-schemas, other-schemas }: {
# Tell Nix what schemas to use.
schemas = flake-schemas.schemas // other-schemas.schemas;
# These flake outputs will now be checked using the schemas above.
packages = ...;
devShells = ...;
};
}
```
## Defining your own schemas
With schemas, we can now teach Nix about the `lib` output mentioned previously. Below is a flake that has a `lib` output.
Similar to `lib` in Nixpkgs, it has a nested structure (for example it provides a function `lists.singleton`).
The flake also has a `schemas.lib` attribute that tells Nix two things:
1. How to list the contents of the `lib` output.
1. To check that every function name follows the camelCase naming convention.
```nix
{
outputs = { self }: {
schemas.lib = {
version = 1;
doc = ''
The `lib` flake output defines Nix functions.
'';
inventory = output:
let
recurse = attrs: {
children = builtins.mapAttrs (attrName: attr:
if builtins.isFunction attr
then
{
# Tell `nix flake show` what this is.
what = "library function";
# Make `nix flake check` enforce our naming convention.
evalChecks.camelCase = builtins.match "^[a-z][a-zA-Z]*$" attrName == [];
}
else if builtins.isAttrs attr
then
# Recurse into nested sets of functions.
recurse attr
else
throw "unsupported 'lib' type")
attrs;
};
in recurse output;
};
lib.id = x: x;
lib.const = x: y: x;
lib.lists.singleton = x: [x];
#lib.ConcatStrings = ...; # disallowed
};
}
```
With this schema, `nix flake show` can now show information about `lib`:
```shell
# nix flake show
git+file:///home/eelco/Determinate/flake-schemas/lib-test
└───lib
├───const: library function
├───id: library function
└───lists
└───singleton: library function
```
While `nix flake check` will now complain if we add a function that violates the naming convention we defined:
```shell
# nix flake check
warning: Evaluation check 'camelCase' of flake output attribute 'lib.ConcatStrings' failed.
```
## What schemas are not
Flake schemas are not a type system for Nix, since that would be a huge project.
They merely provide an _interface_ that enables users to tell Nix how to enumerate and check flake outputs.
For instance, for a [NixOS] configuration, this means using the module system to check that the configuration evaluates correctly; for a Nix package, it just means that the output attribute evaluates to a [derivation].
## Next steps
Flake schemas are new, and they're a valuable expansion of the user experience of [FlakeHub] and [Nix flakes][flakes] in general.
We believe that incorporating schemas into Nix itself will make flakes more broadly valuable and cover use cases that we haven't yet imagined.
We're looking for input from the community to see whether the current schema design covers all use cases.
We've submitted a [pull request][pr] to the [Nix project][nix-repo] that adds schema support to [`nix flake show`][nix-flake-show] and [`nix flake check`][nix-flake-check].
Please take a look!
As future work, schemas will enable us to finally make flakes _configurable_ in a discoverable way: flake schemas can return the configuration options supported by a flake output—for the `nixosConfigurations` output, for example, these would be all the NixOS options—and then the [Nix CLI][cli] can enable users to override options from the command line.
## Conclusion
Flake schemas solve a long-standing problem with flakes: the fact that Nix currently has built-in support for a only small set of output types, which made those outputs more equal than others.
With schemas, all flake output types are on an equal footing.
[cli]: https://zero-to-nix.com/concepts/nix#cli
[derivation]: https://zero-to-nix.com/concepts/derivations
[flakehub]: https://flakehub.com
[flakes]: https://zero-to-nix.com/concepts/flakes
[hydra]: https://github.com/NixOS/hydra
[modules]: https://zero-to-nix.com/concepts/nixos#modules
[nix]: https://nixos.org
[nix-develop]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-develop.html
[nix-flake-check]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-check.html
[nix-flake-show]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-show.html
[nix-repo]: https://github.com/NixOS/nix
[nixos]: https://zero-to-nix.com/concepts/nixos
[nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs
[output]: https://zero-to-nix.com/concepts/flakes#outputs
[packages]: https://zero-to-nix.com/concepts/packages
[pr]: https://github.com/NixOS/nix/pull/8892
[repo]: https://github.com/DeterminateSystems/flake-schemas
---
**Introducing FlakeHub**
Published: August 22, 2023
URL: https://determinate.systems/blog/introducing-flakehub
Today, we at [Determinate Systems][detsys] are extremely excited to announce the release of [**FlakeHub**][flakehub], a platform for discovering and publishing [Nix flakes][flakes].
FlakeHub provides the Nix ecosystem with a variety of new [capabilities](#features):
- The ability to [explore](#explore) the Nix flake landscape.
- [Semantic versioning](#semver) for flakes, including version modifiers like `~` (flexible patch) and `=` (exact match).
- Automated [flake publishing](#publish) with GitHub Actions.
We think that FlakeHub could be a transformative force in the Nix ecosystem and provide a crucial inflection point for flake adoption within and outside of the Nix community.
We can't wait to see what people do with it.
Check out our flake publishing wizard at [flakehub.com/new][wizard] if you want to get started now or read on to learn more.
## What FlakeHub offers \{#features}
### A world of flakes \{#explore}
FlakeHub enables you to explore the current universe of Nix flakes in a variety of new ways.
**Search**.
With FlakeHub, you can search all published flakes by publisher, project, description, and tags.
To activate the FlakeHub search widget you can either click on the magnifying glass icon in the navbar or press Cmd + K on macOS or Ctrl + K on Linux, or
**List all flakes**.
Go to [flakehub.com/flakes][all-flakes] to see a listing of all published flakes.
**Organizations**.
Go to [flakeshub.com/orgs][orgs] to see all the organizations that have published flakes.
**Tags**.
You can use the `/tag/:tag` endpoint to search for flakes by tag.
[flakehub.com/tag/nixos][nixos-tag], for example, shows you all flakes with the `nixos` tag.
### Semantic versioning \{#semver}
Flakes are currently rooted in [revisions].
Revision hashes are valuable because they're highly granular; _any_ change in the contents of the objects in the commit produces a new revision ID.
But Nix, even with flakes, doesn't have a built-in concept of _versions_.
As a refresher, here's what semantic versions look like:
SemVer is popular because it's expressive.
Major, minor, and patch are straightforward markers (hence the "semantic").
The difference between version 1.4.1 and 1.4.2 varies based on the project, and it isn't always clear what should constitute a patch version versus a minor version, but it at least provides _a_ framework for making those judgments.
Revision hashes do not.
In the Nix ecosystem, upgrading a [flake input][input] using [`nix flake update`][nix-flake-update] has thus far meant switching to the most recent Git revision for that [reference][references].
FlakeHub changes that by embedding semantic versioning directly into [flake references][references].
Here's the basic structure:
Here are some example [`nix flake metadata`][nix-flake-metadata] commands that illustrate what FlakeHub makes possible:
```shell
# See the most recent Nixpkgs stable
nix flake metadata "https://flakehub.com/f/NixOS/nixpkgs/*.tar.gz"
# See the most recent Nixpkgs under major version 0.2305
nix flake metadata "https://api.flakehub.com/f/NixOS/nixpkgs/0.2305.tar.gz"
# See the most recent Nixpkgs from NixOS 0.2305, the current stable release
nix flake metadata "https://flakehub.com/f/NixOS/nixpkgs/0.2305.x.tar.gz"
# See the most recent Nixpkgs under minor version 0.2305
nix flake metadata "https://api.flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz"
```
In addition to `*`, FlakeHub also supports the `~` and `=` operators.
This URL denotes the most recent minor version using `~`:
To see the metadata:
```shell
nix flake metadata "https://flakehub.com/f/NixOS/nixpkgs/~0.2305.tar.gz"
```
This URL denotes an exact version using `=`:
To see the metadata:
```shell
nix flake metadata "https://flakehub.com/f/NixOS/nixpkgs/=0.2305.tar.gz"
```
Now let's see how this looks in a `flake.nix` file:
```nix
{
inputs.nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.*.tar.gz";
outputs = { self, nixpkgs }: {
# Use the nixpkgs input here
};
}
```
Now when you run [`nix flake show`][nix-flake-show] in this directory, the `flake.lock` will pin [Nixpkgs] to the most recent revision under 0.2305.
When you run [`nix flake update`][nix-flake-update], it will stay in that series.
### Publish your flakes \{#publish}
You can publish flakes to FlakeHub using [GitHub Actions][actions].
To automatically publish your flake every time you push a new tag, you can add this [workflow config][workflow] to your project:
```yaml
name: Push flake to FlakeHub
on:
push:
tags:
- "v*.*.*"
jobs:
flakehub:
runs-on: ubuntu-22.04
permissions:
id-token: write
contents: read
steps:
- uses: DeterminateSystems/determinate-nix-action@v3
- uses: actions/checkout@v3
- name: Push to FlakeHub
uses: determinatesystems/flakehub-push@main
with:
visibility: "public"
```
To adopt a "rolling" strategy:
```yaml
on:
push:
branches:
- main
jobs:
flakehub:
# Other configs from above
- name: Push to FlakeHub
uses: determinatesystems/flakehub-push@main
with:
rolling: true
visibility: "public"
```
This sets a rolling prefix to be suffixed with the commit count.
`v0.1`, for example, would be suffixed with `.123`, producing a release named `v0.1.123-{revision}`.
We've also provided a handy publishing wizard at [flakehub.com/new][wizard] that walks you through the process step by step.
## You should adopt flakes
Here at Determinate Systems, we are 100% all in on [Nix flakes][flakes] because we firmly believe that they are the future of Nix.
Here are some reasons why:
- Flakes overcome _all_ of the deficiencies of [Nix channels][channels], most importantly surrounding [pinning] to specific revisions of your code via the [`flake.lock`][lock] file.
- Flakes make Nix expressions dramatically more introspectable through commands like [`nix flake show`][nix-flake-show] and [`nix flake metadata`][nix-flake-metadata].
- Flake [references] provide a convenient universal mechanism for specifying the location of a flake.
- Flakes provide standard way to structure Nix [outputs].
- In place of a mélange of files like `default.nix`, `configuration.nix`, and `shell.nix`, flakes are standardized a single `flake.nix` file as the entry point of a project.
And our confidence in flakes isn't just rhetorical.
We've acted on it:
- [The Determinate Nix Installer][nix-installer], our unofficial, still-experimental installer for Nix, enables flakes by default.
- [Zero to Nix][z2n], an opinionated learning resource for Nix that we created, is centered around flakes and actively promotes beginning your Nix journey with flakes rather than [channels].
- We created both [Flake Checker][checker], a tool that performs "health checks" on your [`flake.lock`][lock] files, and the [`update-flake-lock`][update] Github Action, which automates `flake.lock` updates for GitHub projects.
FlakeHub is our most ambitious salvo yet and part of a [broader story][better] that guides everything we do as a company.
## Current Nix practices
SemVer has become standard practice in most programming language communities: JavaScript, Python, Rust, Java, Ruby, and on down the line.
Noticeably absent from this list: **Nix**.
Prior to the introduction of [flakes], most Nix development used [Nixpkgs] as the only external Nix input.
Typically, developers would subscribe to a [channel][channels] of Nixpkgs, like `22.11` or `23.05`, continuously update to the most recent revision in that channel, and then switch channels every six months.
The introduction of flakes began to change things.
Because flakes provide primitives to compose and [pin][pinning] your Nix inputs, we began to see a wider variety of inputs in Nix projects—a decentralization of the Nix universe.
But even with flakes, Nix development still typically involves one of two approaches to handling Nix flake inputs:
- Give me the most recent revision in this specific branch.
And there are only a few prominent flakes, like [Nixpkgs], where branches, such as `nixpkgs-unstable`, are used.
More often the approach is...
- Just give me the most recent revision.
The consequence is that operations like [`nix flake update`][nix-flake-update] are too often fraught with peril.
You run it, you hope for the best, you try to fix what breaks, and you steel yourself to do the same over and over again in the future.
We hope that FlakeHub and the introduction of [SemVer](#semver) to [flakes] enables the Nix community to move beyond this state of affairs and into harmony with other languages.
We want `nix flake update` to feel a lot more like housekeeping and a lot less like a venturesome leap into the unknown.
## How it works \{#how}
In order to make semantic versions work in FlakeHub, we submitted a change to Nix that enables it to handle arbitrary URL endpoints.
If the endpoint returns an [HTTP 301][301] status code and a `Link` header, Nix fetches the flake tarball from that URL and records that URL in the `flake.lock` file.
See [the FlakeHub docs][upgrade-nix] for information on which versions of Nix support this.
Take this `flake.nix` file:
```nix
{
inputs.nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2305.490090.tar.gz";
# Flake outputs here
}
```
If you run a `nix flake` command like [`nix flake show`][nix-flake-show] in this directory, the generated `flake.lock` looks like this:
```json
{
"nodes": {
"nixpkgs": {
"locked": {
"narHash": "sha256-7el+r373PubFExJSr/FA4wwr66ekBn4afDJuEiuefO8=",
"rev": "475d5ae2c4cb87b904545bdb547af05681198fcc",
"revCount": 490311,
"type": "tarball",
"url": "https://api.flakehub.com/f/pinned/NixOS/nixpkgs/0.2305.490311%2Brev-475d5ae2c4cb87b904545bdb547af05681198fcc/018a1928-9148-72c2-b387-ee7e2286c1b9/source.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://api.flakehub.com/f/NixOS/nixpkgs/0.2305.%2A.tar.gz"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}
```
As you can see, Nix has resolved the URL into a specific tarball reference using the FlakeHub server's HTTP response.
FlakeHub's implementation includes not just Git revision information but also a UUID in the tarball URL.
And FlakeHub is just one implementation.
With this feature now in Nix, you can build your own HTTP servers to serve flakes according to whatever scheme you wish, SemVer or not.
## Room for improvement
There are a few areas where FlakeHub isn't quite where we want it to be yet:
- **CLI tool**.
In the coming weeks, we intend to introduce a CLI tool that enables you to perform a variety of actions against the FlakeHub API, such as search.
- **Private flakes**.
Upon launch, all the flakes on FlakeHub are either [public or unlisted][visibility]; unlisted flakes can be used by Nix but don't show up on the website.
Down the road, we want to enable you to make flakes available only to people within your org.
## Implications
We're confident that FlakeHub will be a major boon to flakes adoption and to the Nix ecosystem more broadly.
A dedicated home for flakes will enable people to see which flakes exist and add them to their daily workflows more easily than ever before—and within those workflows, the update path will be predictable in a way that we haven't yet seen in the world of Nix.
Whether SemVer takes hold in Nix is up to you.
But we're confident that adoption will happen swiftly now that Nix users have the means to adopt it.
Take a look at [**flakehub.com**][flakehub], browse the docs at [docs.determinate.systems][docs], and [publish your flake][wizard] today.
Or join us [on Discord][discord] if you have questions or problems or just want to connect with other Nix users.
[301]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/301
[actions]: https://github.com/features/actions
[all-flakes]: https://flakehub.com/flakes
[better]: /blog/we-want-to-make-nix-better
[channels]: https://zero-to-nix.com/concepts/channels
[checker]: https://github.com/DeterminateSystems/flake-checker
[detsys]: https://determinate.systems
[discord]: https://discord.gg/invite/a4EcQQ8STr
[docs]: https://docs.determinate.systems
[flakehub]: https://flakehub.com
[flakes]: https://zero-to-nix.com/concepts/flakes
[input]: https://zero-to-nix.com/concepts/flakes#inputs
[nix-installer]: https://github.com/determinateSystems/nix-installer
[lock]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-lock.html
[nix-flake-metadata]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-metadata.html
[nix-flake-show]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-show.html
[nix-flake-update]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-update.html
[nixos-tag]: https://flakehub.com/tag/nixos
[nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs
[orgs]: https://flakehub.com/orgs
[outputs]: https://zero-to-nix.com/concepts/flakes#outputs
[pinning]: https://zero-to-nix.com/concepts/pinning
[references]: https://zero-to-nix.com/concepts/flakes#references
[revisions]: https://zero-to-nix.com/concepts/flakes#revisions
[update]: https://github.com/determinateSystems/update-flake-lock
[upgrade-nix]: https://docs.determinate.systems/determinate-nix#determinate-nixd-upgrade
[visibility]: https://docs.determinate.systems/flakehub/concepts/visibility
[wizard]: https://flakehub.com/new
[workflow]: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions
[z2n]: https://zero-to-nix.com
---
**Instrumenting Axum projects**
Published: July 20, 2023
URL: https://determinate.systems/blog/instrumenting-axum
Recently I was helping someone get bootstrapped on their Axum project, and they were getting a bit frustrated with the lack of context about the goings-on in their application.
While Axum and its ecosystem are well instrumented, by default most of it is not well surfaced for beginners.
It can be a bit intimidating figuring out how to put the pieces together and use them.
Once we set up [`tracing`] and [`color_eyre`] our logging and errors went from looking like this:
```
Listening on [::]:8080
Got an error request on `/error`
Got request for `/favicon.ico`, but no route was found.
Got a homepage request on `/`
```
To something with a little bit more context:
```
INFO Listening on [::]:8080
INFO request:get_error: Got an error request uri=/error method=GET source=::1
ERROR request:
0: Whoopsies
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
0: demo::routes::get_error
at src/routes.rs:11
1: demo::trace_layer::request with uri=/error method=GET source=::1
at src/trace_layer.rs:24
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets. uri=/error method=GET source=::1
INFO request:get_home: Got a homepage request uri=/ method=GET source=::1
INFO request:fallback_404: Got request, but no route was found. uri=/favicon.ico method=GET source=::1
```
While this isn't a complete and fully mature solution suitable for an industrial application, it is a good starting point for them that smoothly evolve into something more mature.
Often the first place I start with a binary project is the command line interface, so we'll start there.
## Making instrumentation tuneable
Instrumentation can have many knobs, this project used [`clap`], so we took the existing `Cli` struct and added an `instrumentation` field:
```rust
// src/cli/mod.rs
mod instrumentation;
mod logger;
use clap::Parser;
use std::net::{IpAddr, Ipv6Addr, SocketAddr};
#[derive(Parser)]
pub(crate) struct Cli {
#[clap(long, env = "DEMO_BIND", default_value_t = SocketAddr::new(
IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0, 0, 0)), 8080)
)]
pub(crate) bind: SocketAddr,
#[clap(flatten)]
pub(crate) instrumentation: instrumentation::Instrumentation,
}
```
Using `#[clap(flatten)]` here means we can define all our instrumentation options in a separate struct, while still appearing in the `--help` as one might expect:
```
❯ cargo run -- --help
Finished dev [unoptimized + debuginfo] target(s) in 0.04s
Running `target/debug/demo --help`
Usage: demo [OPTIONS]
Options:
--bind
[env: DEMO_BIND=]
[default: [::]:8080]
-v, --verbose...
Enable debug logs, -vv for trace
[env: DEMO_VERBOSITY=]
--logger
Which logger to use
[env: DEMO_LOGGER=]
[default: compact]
[possible values: compact, full, pretty, json]
--log-directive [...]
Tracing directives
See https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#directives
[env: DEMO_LOG_DIRECTIVES=]
-h, --help
Print help (see a summary with '-h')
```
Tracing offers use the choice of several built in logging styles, or even our own logging style if we so choose!
It's straightforward to pass on this choice to the user, though we do need to create `Logger` enum to represent that choice:
```rust
// src/cli/logger.rs
#[derive(Clone, Default, Debug, clap::ValueEnum)]
pub(crate) enum Logger {
#[default]
Compact,
Full,
Pretty,
Json,
}
impl std::fmt::Display for Logger {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let logger = match self {
Logger::Compact => "compact",
Logger::Full => "full",
Logger::Pretty => "pretty",
Logger::Json => "json",
};
write!(f, "{}", logger)
}
}
```
The `Logger` can then get used in our `Instrumentation` struct, which we'll build up over the next few code blocks:
```rust
// src/cli/instrumentation.rs
use color_eyre::eyre::WrapErr;
use std::{error::Error, io::IsTerminal};
use tracing::Subscriber;
use tracing_subscriber::{
filter::Directive,
layer::{Layer, SubscriberExt},
registry::LookupSpan,
util::SubscriberInitExt,
EnvFilter,
};
use super::logger::Logger;
#[derive(clap::Args, Debug, Default)]
pub(crate) struct Instrumentation {
/// Enable debug logs, -vv for trace
#[clap(
short = 'v',
env = "DEMO_VERBOSITY",
long, action = clap::ArgAction::Count,
global = true
)]
pub verbose: u8,
/// Which logger to use
#[clap(
long,
env = "DEMO_LOGGER",
default_value_t = Default::default(),
global = true
)]
pub(crate) logger: Logger,
/// Tracing directives
///
/// See https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#directives
#[clap(long = "log-directive", global = true, env = "DEMO_LOG_DIRECTIVES", value_delimiter = ',', num_args = 0..)]
pub(crate) log_directives: Vec,
}
impl Instrumentation {
pub(crate) fn log_level(&self) -> String {
match self.verbose {
0 => "info",
1 => "debug",
_ => "trace",
}
.to_string()
}
// (continued below)
}
```
This interface offers users the ability to specify 'traditional' verbosity options such as `-v` or, my favorite, `-vvvvvvv` through the use of [`clap::ArgAction::Count`].
It also permits the use of more [`tracing`]-specific options like the `Logger` we created above, or the option of any number of [`tracing_subscriber::filter::Directive`]. We'll explore those a bit together once we get things running.
We can then attach some functions to this struct that set up a [`Registry`][`tracing_subscriber::registry::Registry`] which stores [span][`tracing::span`] data, such as the data defined in [`#[tracing::instrument]`][`tracing::instrument`] calls.
In a `setup()` function we'll build a [`Registry`][`tracing_subscriber::registry::Registry`] and compose it with several layers, including a [`ErrorLayer`][`tracing_error::ErrorLayer`] and an [`EnvFilter`][`tracing_subscriber::filter::EnvFilter`] layer we configure from the knobs made available in the `Cli` struct (as well as some conventional environment variables):
```rust
impl Instrumentation {
// (continued)
pub(crate) fn setup(&self) -> color_eyre::Result<()> {
let filter_layer = self.filter_layer()?;
let registry = tracing_subscriber::registry()
.with(filter_layer)
.with(tracing_error::ErrorLayer::default());
// `try_init` called inside `match` since `with` changes the type
match self.logger {
Logger::Compact => {
registry.with(self.fmt_layer_compact()).try_init()?
}
Logger::Full => {
registry.with(self.fmt_layer_full()).try_init()?
}
Logger::Pretty => {
registry.with(self.fmt_layer_pretty()).try_init()?
}
Logger::Json => {
registry.with(self.fmt_layer_json()).try_init()?
}
}
Ok(())
}
pub(crate) fn filter_layer(&self) -> color_eyre::Result {
let mut filter_layer = match EnvFilter::try_from_default_env() {
Ok(layer) => layer,
Err(e) => {
// Catch a parse error and report it, ignore a missing env
if let Some(source) = e.source() {
match source.downcast_ref::() {
Some(std::env::VarError::NotPresent) => (),
_ => return Err(e).wrap_err_with(|| "parsing RUST_LOG directives"),
}
}
// If the `--log-directive` is specified, don't set a default
if self.log_directives.is_empty() {
EnvFilter::try_new(&format!(
"{}={}",
env!("CARGO_PKG_NAME").replace('-', "_"),
self.log_level()
))?
} else {
EnvFilter::try_new("")?
}
}
};
for directive in &self.log_directives {
let directive_clone = directive.clone();
filter_layer = filter_layer.add_directive(directive_clone);
}
Ok(filter_layer)
}
// (continued below)
}
```
Then we can go ahead and define the various format layers, along with any [customization](https://docs.rs/tracing-subscriber/latest/tracing_subscriber/fmt/struct.Layer.html#) we wanted to do.
```rust
impl Instrumentation {
// (continued)
pub(crate) fn fmt_layer_full(&self) -> impl Layer
where
S: Subscriber + for<'span> LookupSpan<'span>,
{
tracing_subscriber::fmt::Layer::new()
.with_ansi(std::io::stderr().is_terminal())
.with_writer(std::io::stderr)
}
pub(crate) fn fmt_layer_pretty(&self) -> impl Layer
where
S: Subscriber + for<'span> LookupSpan<'span>,
{
tracing_subscriber::fmt::Layer::new()
.with_ansi(std::io::stderr().is_terminal())
.with_writer(std::io::stderr)
.pretty()
}
pub(crate) fn fmt_layer_json(&self) -> impl Layer
where
S: Subscriber + for<'span> LookupSpan<'span>,
{
tracing_subscriber::fmt::Layer::new()
.with_ansi(std::io::stderr().is_terminal())
.with_writer(std::io::stderr)
.json()
}
pub(crate) fn fmt_layer_compact(&self) -> impl Layer
where
S: Subscriber + for<'span> LookupSpan<'span>,
{
tracing_subscriber::fmt::Layer::new()
.with_ansi(std::io::stderr().is_terminal())
.with_writer(std::io::stderr)
.compact()
.without_time()
.with_target(false)
.with_thread_ids(false)
.with_thread_names(false)
.with_file(false)
.with_line_number(false)
}
}
```
Later, your project may choose to integrate something like [`tracing_opentelemetry`] and route that data to a service like [Honeycomb].
## Building up `Error`s
[`axum`] allows us to define routes that can return errors, if that error implements [`axum::response::IntoResponse`]:
```rust
async fn some_route() -> Result {
Ok("I work ok!")
}
```
Unfortunately, this does not work out of the box with [`eyre::Report`].
It's also forbidden for us (as `demo`) to implement [`IntoResponse`][`axum::response::IntoResponse`] for [`Report`][`eyre::Report`] due to the [Orphan Rule](https://rust-lang.github.io/chalk/book/clauses/coherence.html), so we must create a wrapper. While we do that, we might as well create a crate-specific `DemoError` as well, which can be later used to define user-facing error messages and status codes.
```rust
// src/error.rs
use axum::{
http::StatusCode,
response::{IntoResponse, Response},
};
pub type Result = color_eyre::Result;
// A generic error report
// Produced via `Err(some_err).wrap_err("Some context")`
// or `Err(color_eyre::eyre::Report::new(SomeError))`
pub struct Report(color_eyre::Report);
impl std::fmt::Debug for Report {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
self.0.fmt(f)
}
}
impl From for Report
where
E: Into,
{
fn from(err: E) -> Self {
Self(err.into())
}
}
// Tell axum how to convert `Report` into a response.
impl IntoResponse for Report {
fn into_response(self) -> Response {
let err = self.0;
let err_string = format!("{err:?}");
tracing::error!("{err_string}");
if let Some(err) = err.downcast_ref::() {
return err.response()
}
// Fallback
(
StatusCode::INTERNAL_SERVER_ERROR,
"Something went wrong".to_string(),
)
.into_response()
}
}
#[derive(thiserror::Error, Debug)]
pub(crate) enum DemoError {
#[error("A spooky thing happened")]
Spooky,
}
// Tell axum how to convert `DemoError` into a response.
impl DemoError {
fn response(&self) -> Response {
match self {
Self::Spooky => (
StatusCode::IM_A_TEAPOT,
"A user-facing message about a Spooky".to_string(),
)
.into_response(),
}
}
}
```
Now we can use that new `Result` in our routes, and return errors either ad-hoc via [`eyre::eyre`], or structured through the error type we created with [`thiserror`].
> This is also a good opportunity to decorate your routes with a [`tracing::instrument`] attribute.
>
> Consider using `skip_all` and explicitly defining any fields if you are dealing complex route arguments!
> Eg. `#[tracing::instrument(skip_all, fields(uid = context.user.uid))]`.
```rust
use crate::error::Result;
use axum::response::IntoResponse;
use color_eyre::eyre::eyre;
#[tracing::instrument]
pub(crate) async fn get_error() -> Result {
tracing::info!("Got an error request");
// Bang!!!
Err(eyre!("Whoopsies"))?;
Ok(())
}
```
At the start, and during prototyping, you can use [`eyre::eyre`] to write ad-hoc errors.
As the application grows, you'll be able to work with [`thiserror`] and use [`downcast_ref`](https://docs.rs/eyre/latest/eyre/struct.Report.html#method.downcast_ref) (like we did in the `instrumentation.rs` code) and other tools available in [`eyre::Report`] to respond with a more structured and informative messages to the web browser.
```rust
use crate::error::Result;
use axum::response::IntoResponse;
// After adding a `Spooky` variant to `DemoError` in `src/error.rs`:
//
// #[derive(thiserror::Error, Debug)]
// enum DemoError {
// #[error("A spooky thing happened")]
// Spooky
// }
#[tracing::instrument]
pub(crate) async fn get_demo_error() -> Result {
tracing::info!("Got an error request");
// Bang!!!
Err(DemoError::Spooky)?;
Ok(())
}
```
## Putting the pieces together
After painting our user interface and performing the ritualistic type dancing, it's finally time to update `main()` with the code to get everything working.
At the top of `main()` we want to install [`color_eyre`]'s error hooks, so that any errors after that are fully styled and integrated.
After that, we can call `setup()` on the `Instrumentation` instance created by [`clap`] to initialize tracing.
In order to add the appropriate spans to requests, [`tower_http::trace::TraceLayer`] should be added to the [`axum::routing::Router`]:
```rust
// src/main.rs
mod cli;
mod error;
mod routes;
mod trace_layer;
use crate::{cli::Cli, error::Result};
use axum::routing::get;
use clap::Parser;
use std::{io::IsTerminal, net::SocketAddr, process::ExitCode};
use tower_http::trace::TraceLayer;
#[tokio::main]
async fn main() -> Result {
color_eyre::config::HookBuilder::default()
.theme(if !std::io::stderr().is_terminal() {
// Don't attempt color
color_eyre::config::Theme::new()
} else {
color_eyre::config::Theme::dark()
})
.install()?;
let cli = Cli::parse();
cli.instrumentation.setup()?;
let trace_layer = TraceLayer::new_for_http()
.make_span_with(trace_layer::trace_layer_make_span_with)
.on_request(trace_layer::trace_layer_on_request)
.on_response(trace_layer::trace_layer_on_response);
let app = axum::Router::new()
.route("/", get(routes::get_home))
.route("/errors", get(routes::get_error))
.route("/errors/demo", get(routes::get_demo_error))
.fallback(routes::fallback_404)
.layer(trace_layer);
tracing::info!("Listening on {}", cli.bind);
axum::Server::bind(&cli.bind)
.serve(app.into_make_service_with_connect_info::())
.await?;
Ok(ExitCode::SUCCESS)
}
```
[`TraceLayer`][`tower_http::trace::TraceLayer`] offers opportunities to attach our own functions where we can create spans or emit events.
There exist defaults ([`tower_http::trace::{DefaultMakeSpan, DefaultOnRequest}`][`tower_http::trace`]) which can be modified using a builder API, these have the rather unfortunate quality of being part of the `tower_http` crate, not our own, so because the defaults we configured in the `Instrumentation::filter_layer()` function they won't normally be enabled.
Defining our own functions allows us to build our own base span, as well as ensure these spans are part of our own crate.
If your project opts to use the defaults, consider altering `Instrumentation::filter_layer()` to also set some default for `tower_http`.
```rust
// src/trace_layer.rs
use axum::{body::BoxBody, extract::ConnectInfo, response::Response};
use hyper::{Body, Request};
use std::{net::SocketAddr, time::Duration};
use tracing::Span;
pub(crate) fn trace_layer_make_span_with(request: &Request) -> Span {
tracing::error_span!("request",
uri = %request.uri(),
method = %request.method(),
// This is not particularly robust, but suitable for a demo
// You'll need to change this if you deploy behind a proxy
// (eg the `X-forwarded-for` header)
source = request.extensions()
.get::>()
.map(|connect_info|
tracing::field::display(connect_info.ip().to_string()),
).unwrap_or_else(||
tracing::field::display(String::from(""))
),
// Fields must be defined to be used, define them as empty if they populate later
status = tracing::field::Empty,
latency = tracing::field::Empty,
)
}
pub(crate) fn trace_layer_on_request(_request: &Request, _span: &Span) {
tracing::trace!("Got request")
}
pub(crate) fn trace_layer_on_response(
response: &Response,
latency: Duration,
span: &Span,
) {
span.record(
"latency",
tracing::field::display(format!("{}μs", latency.as_micros())),
);
span.record("status", tracing::field::display(response.status()));
tracing::trace!("Responded");
}
```
We can create a few test routes a fallback route, one reporting no error, another reporting an [`eyre::eyre`] based error, and a route reporting a `DemoError`.
Due to our error handling code we can use `DemoError` variants to return specific, user-facing messages and status codes, while more ad-hoc or developer/operator facing errors can be [`eyre::eyre`] based.
```rust
// src/routes.rs
use crate::error::{Result, DemoError};
use axum::response::IntoResponse;
use color_eyre::eyre::eyre;
#[tracing::instrument]
pub(crate) async fn get_home() -> Result {
tracing::info!("Got a homepage request");
Ok("Welcome to my super cute home page")
}
#[tracing::instrument]
pub(crate) async fn get_error() -> Result {
tracing::info!("Got an error request");
Err(eyre!("Whoopsies"))?;
Ok(())
}
#[tracing::instrument]
pub(crate) async fn get_demo_error() -> Result {
tracing::info!("Got an error request");
Err(DemoError::Spooky)?;
Ok(())
}
#[tracing::instrument(skip_all)]
pub(crate) async fn fallback_404() -> Result {
tracing::info!("Got request, but no route was found.");
Ok("You failed to find my super cute home page")
}
```
If you're following along you may have run `cargo add` for some of the dependencies above, the specific examples shown utilized these features and crates:
```toml
# Cargo.toml
[package]
name = "demo"
version = "0.1.0"
edition = "2021"
[dependencies]
axum = "0.6"
clap = { version = "4.3", features = ["derive", "env"] }
color-eyre = { version = "0.6", default-features = false, features = [ "issue-url", "tracing-error", "capture-spantrace", "color-spantrace" ] }
hyper = "0.14"
thiserror = "1"
tokio = { version = "1", features = ["full"] }
tower = "0.4"
tower-http = { version = "0.4", features = ["trace"] }
tracing = "0.1"
tracing-error = "0.2"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
```
One feature you may wish to enable the `track-caller` on [`color-eyre`][`color_eyre`] package, it will show the locations of errors in the output using the [`track_caller`] attribute.
The reported location is not always accurate, but often it's helpful.
## Going for a test run
At this point, we can run our binary and visit the homepage:
```bash
❯ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.05s
Running `target/debug/demo`
INFO Listening on [::]:8080
INFO request:get_home: Got a homepage request uri=/ method=GET source=::1
INFO request:fallback_404: Got request, but no route was found. uri=/favicon.ico method=GET source=::1
```
We can run the application again with different directives and get some more info from one of the packages, for example [`hyper`] which does the bulk of the HTTP work:
```bash
❯ cargo run -- --logger compact --log-directive demo=trace --log-directive hyper=trace
Finished dev [unoptimized + debuginfo] target(s) in 0.05s
Running `target/debug/demo --logger compact --log-directive demo=trace --log-directive hyper=trace`
INFO Listening on [::]:8080
TRACE Conn::read_head
TRACE received 454 bytes
TRACE parse_headers: Request.parse bytes=454
TRACE parse_headers: Request.parse Complete(454)
DEBUG parsed 12 headers
DEBUG incoming body is empty
TRACE request: Got request uri=/ method=GET source=::1
INFO request:get_home: Got a homepage request uri=/ method=GET source=::1
TRACE request: Responded uri=/ method=GET source=::1 latency=175μs status=200 OK
TRACE encode_headers: Server::encode status=200, body=Some(Known(34)), req_method=Some(GET)
TRACE sized write, len = 34
TRACE buffer.queue self.len=117 buf.len=34
# ...
```
That's **a lot of output**! We can also use [log directives][`tracing_subscriber::filter::EnvFilter`] to drill down into specific parts of the code, this can be quite useful for debugging. Here we isolate [`hyper`] events to `parse_headers`:
```bash
❯ cargo run -- --log-directive hyper[parse_headers]=trace --log-directive demo=info
Finished dev [unoptimized + debuginfo] target(s) in 0.05s
Running `target/debug/demo --log-directive 'hyper[parse_headers]=trace' --log-directive demo=info`
INFO Listening on [::]:8080
TRACE parse_headers: Request.parse bytes=464
TRACE parse_headers: Request.parse Complete(464)
INFO request:get_demo_error: Got an error request uri=/error/demo method=GET source=::1
ERROR request:
0: A spooky thing happened
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
0: demo::routes::get_demo_error
at src/routes.rs:20
1: demo::trace_layer::request with uri=/error/demo method=GET source=::1
at src/trace_layer.rs:24
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets. uri=/error/demo method=GET source=::1
```
Or how about only logging requests for a specific URL? We can specify a filter like `demo[request{uri=/}]=trace` and visit several pages, observing we only see logs for the one we filtered for:
```bash
❯ cargo run -- --log-directive demo[request{uri=/}]=trace
Compiling demo v0.1.0 (/home/ana/git/determinatesystems/axum-with-tracing-and-eyre)
Finished dev [unoptimized + debuginfo] target(s) in 2.25s
Running `target/debug/demo --log-directive 'demo[request{uri=/}]=trace'`
TRACE request: Got request uri=/ method=GET source=::1
INFO request:get_home: Got a homepage request uri=/ method=GET source=::1
TRACE request: Responded uri=/ method=GET source=::1 latency=333μs status=200 OK
```
Of the different logger options, my personal favorite is the [`pretty`][`tracing_subscriber::fmt::format::Pretty`] logger which creates a cozy, human readable experience:
```bash
❯ cargo run -- --logger pretty
Finished dev [unoptimized + debuginfo] target(s) in 0.04s
Running `target/debug/demo --logger pretty`
2023-07-19T21:56:16.481962Z INFO demo: Listening on [::]:8080
at src/main.rs:38
2023-07-19T21:56:20.084723Z INFO demo::routes: Got a homepage request
at src/routes.rs:7
in demo::routes::get_home
in demo::trace_layer::request with uri: /, method: GET, source: ::1
2023-07-19T21:56:20.131037Z INFO demo::routes: Got request, but no route was found.
at src/routes.rs:40
in demo::routes::fallback_404
in demo::trace_layer::request with uri: /favicon.ico, method: GET, source: ::1
```
The tracing spans also appear in our errors, let's take a look at those. We can observe these errors when we visit the respective `/error` urls:
```bash
❯ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.04s
Running `target/debug/demo`
INFO Listening on [::]:8080
INFO request:get_error: Got an error request uri=/error method=GET source=::1
ERROR request:
0: Whoopsies
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
0: demo::routes::get_error
at src/routes.rs:12
1: demo::trace_layer::request with uri=/error method=GET source=::1
at src/trace_layer.rs:24
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets. uri=/error method=GET source=::1
INFO request:fallback_404: Got request, but no route was found. uri=/favicon.ico method=GET source=::1
INFO request:get_demo_error: Got an error request uri=/error/demo method=GET source=::1
ERROR request:
0: A spooky thing happened
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
0: demo::routes::get_demo_error
at src/routes.rs:27
1: demo::trace_layer::request with uri=/error/demo method=GET source=::1
at src/trace_layer.rs:24
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets. uri=/error/demo method=GET source=::1
INFO request:fallback_404: Got request, but no route was found. uri=/favicon.ico method=GET source=::1
```
Visiting `/error/demo` we can observe the browser returning `A spooky occured` as we intended.
## Conclusion
The combination of [`thiserror`], [`tracing`], and [`color_eyre`] provides a solid starting point for a budding project. The ability to instrument code with spans, see those spans in errors, then filter based on those spans in logging messages enables greater insight while diagnosing issues and authoring new features. As your project grows, these same tools offer a smooth path to adoption of standards like [OpenTelemetry].
While researching this article I bumped into the really lovely article by [@carlosmv](https://carlosmv.hashnode.dev/adding-logging-and-tracing-to-an-axum-app-rust) about a many of the same ideas! They go in detail about how to use [`tracing_appender`] and some other things which are not covered here, but don't discuss error handling as much. If you want to dig deeper into this topic, that article is a wonderful place to visit after this!
A few years ago I gave [a recorded talk at TremorCon](https://www.youtube.com/watch?v=ZC7fyqshun8) which discussed [`tracing`] and [`color_eyre`], and this article expands on several of the concepts mentioned there.
[`axum::response::IntoResponse`]: https://docs.rs/axum/0.6/axum/response/trait.IntoResponse.html
[`axum::routing::Router`]: https://docs.rs/axum/0.6/axum/routing/struct.Router.html
[`axum`]: https://docs.rs/axum/0.6/axum/
[`clap::ArgAction::Count`]: https://docs.rs/clap/4.3/clap/enum.ArgAction.html#variant.Count
[`clap`]: https://docs.rs/clap/4.3/clap/
[`color_eyre`]: https://docs.rs/color-eyre/0.6/color_eyre/
[`eyre::eyre`]: https://docs.rs/eyre/0.6/eyre/macro.eyre.html
[`eyre::Report`]: https://docs.rs/eyre/0.6/eyre/struct.Report.html
[`hyper`]: https://docs.rs/hyper/0.14/hyper/
[`thiserror`]: https://docs.rs/thiserror/1/thiserror/
[`tower_http::trace::TraceLayer`]: https://docs.rs/tower-http/0.4.1/tower_http/trace/struct.TraceLayer.html
[`tower_http::trace`]: https://docs.rs/tower-http/0.4/tower_http/trace/index.html
[`tracing_appender`]: https://docs.rs/tracing-appender/latest/tracing_appender/
[`tracing_error::ErrorLayer`]: https://docs.rs/tracing-error/0.2/tracing_error/struct.ErrorLayer.html
[`tracing_opentelemetry`]: https://docs.rs/tracing-opentelemetry/latest/tracing_opentelemetry/
[`tracing_subscriber::filter::Directive`]: https://docs.rs/tracing-subscriber/0.3/tracing_subscriber/filter/struct.Directive.html
[`tracing_subscriber::filter::EnvFilter`]: https://docs.rs/tracing-subscriber/0.3/tracing_subscriber/filter/struct.EnvFilter.html
[`tracing_subscriber::fmt::format::Pretty`]: https://docs.rs/tracing-subscriber/0.3/tracing_subscriber/fmt/format/struct.Pretty.html
[`tracing_subscriber::registry::Registry`]: https://docs.rs/tracing-subscriber/0.3/tracing_subscriber/registry/struct.Registry.html
[`tracing::instrument`]: https://docs.rs/tracing/0.1/tracing/attr.instrument.html
[`tracing::span`]: https://docs.rs/tracing/latest/tracing/span/index.html
[`tracing`]: https://docs.rs/tracing/0.1/tracing/
[`track_caller`]: https://rustc-dev-guide.rust-lang.org/backend/implicit-caller-location.html
[Honeycomb]: https://www.honeycomb.io/
[OpenTelemetry]: https://opentelemetry.io/
---
**Introducing the Magic Nix Cache**
Published: June 26, 2023
URL: https://determinate.systems/blog/magic-nix-cache
We at [Determinate Systems][detsys], along with our collaborator [Zhaofeng Li][zhaofeng], are thrilled to announce the release of the [**Magic Nix Cache**][action], a [GitHub Action][actions] that can dramatically speed up your [Nix]-related workflows.
You can use it today—no signup, setup, or cost—by adding just one line to your YAML configuration:
```yaml
- uses: DeterminateSystems/magic-nix-cache-action@v2
```
Here's a more complete example:
```yaml
jobs:
build:
steps:
- uses: actions/checkout@v4
- name: Install Nix
uses: DeterminateSystems/determinate-nix-action@v3
- name: Run the Magic Nix Cache
uses: DeterminateSystems/magic-nix-cache-action@v2
- name: Build server and client
run: |
nix build .#server
nix build .#client
```
We're confident that its power and ease of use will provide immediate benefits to the Nix community at large.
## Why caching is important
We at [Determinate Systems][detsys] use [Nix] for all of our [GitHub Actions][actions] builds.
Synchronizing environments [across dev, prod, and CI][dev] is, after all, one of Nix's major [strong points][features].
But let's be frank: Nix builds can be quite slow.
Nix always [realises] a [derivation]'s entire dependency tree when it builds something, and that tree can be quite vast.
Fortunately, Nix is extremely good at [caching].
Because everything stored in the [Nix store][store] has a [content-derived hash][hash] in the store path, Nix can quickly determine if something needs to be built or if it can be fetched from a known [binary cache][binary] instead.
Caching can cut build times by orders of magnitude (depending on the context).
To reap the benefits of caching in a CI environment like GitHub Actions, you previously needed to either:
1. Run your own binary cache backed by a storage platform like [Amazon S3][s3] or S3-compatible [Minio].
1. Use a dedicated SaaS service like [Cachix].
The Magic Nix Cache provides a new option that doesn't require you to deploy any new infrastructure or sign up for an external service.
## How it works
The Magic Nix Cache relies on a daemon called [`magic-nix-cache`][magic-nix-cache], which is written in [Rust].
This daemon acts just like a standard [binary cache server][binary] from the standpoint of Nix, but with the important difference that it writes to the [GitHub Actions Cache][actions-cache] using the [Github Actions Cache API][cache-api] instead of writing to disk, as a standard binary cache would.
When you add the Magic Nix Cache to an Actions workflow, here's what happens:
- When the Action is triggered, it first downloads and runs the binary for the `magic-nix-cache` daemon.
The daemon then fetches a list of what's currently stored in the GitHub Actions Cache.
- The rest of your workflow logic, including all of your Nix commands, proceeds.
- When the workflow is finished, the `magic-nix-cache` daemon calculates a list of Nix store paths that haven't yet been written to the GitHub Actions Cache and writes them.
With this approach, the Magic Nix Cache caches _everything_ written to the Nix store during your workflow run.
That includes your Nix sources (including bulkier things like revisions of [Nixpkgs]), [development environments][dev-env], all the intermediate dependencies from `nix build` invocations, [Docker] images you build with Nix, and more.
And it does this automatically, without requiring a workflow step of this sort:
```yaml
# NOT necessary so don't do this ❌
- name: Cache Nix artifacts
run: ./complex-nix-cache-script.sh
```
### Security
The Magic Nix Cache provides the same [security guarantees][security] as the [Actions Cache API][cache-api], which includes **cache isolation** across different branches and tags.
Concretely, this means that the cache for a project can never be polluted by a malicious pull request because workflow runs never restore caches created for child or sibling branches.
If someone submits a pull request for a branch called `uh-oh-bitcoin-miner-incoming`, for example, the Magic Nix Cache _will_ indeed cache the Nix artifacts for that workflow but nothing from that cache will ever be used by your other branches, including your default branch (such as `main`).
### What about rate limits?
Rate limits are a persistent concern for GitHub Actions users, and many of us have been bitten by workflow runs that fail by inadvertently exceeding those limits.
The Magic Nix Cache _is_ subject to the rate limits of the [Actions Cache API][cache-api] and large projects or builds may run up against those constraints.
If that happens, you'll see log messages like this:
```log
error: unable to download 'http://127.0.0.1:37515/<...>': HTTP error 418
response body:
GitHub API error: API error (429 Too Many Requests): StructuredApiError { message: "Request was blocked due to exceeding usage of resource 'Count' in namespace ''." }
```
Fortunately, the Magic Nix Cache handles any rate limiting gracefully.
If your workflow exceeds the limit when it's pulling Nix dependencies from your Actions cache, Nix will build those dependencies rather than fetching them from the cache.
And if your workflow exceeds the limit when pushing to the cache, the Magic Nix Cache will stop caching and wait for the next workflow run to push those store paths to the cache.
Rate limiting may thus increase build times on specific workflow runs but it will _not_ cause your runs to fail.
## Drawbacks
Because the Magic Nix Cache uses the [Actions Cache API][cache-api], the artifacts it caches are available only to workflow runs for your project.
This means that:
- **Caches are shared across workflow runs but not across GitHub repos**.
We suspect, however, that any potential speed benefits of cross-project caching would be marginal at best.
- **Caches aren't publicly available**.
If you'd like your Actions workflows to be used by members of your org or Nix users more broadly, you'll need to push your Nix artifacts to a standard [binary cache][binary].
But there's nothing preventing you from using the Magic Nix Cache to speed up future Nix workflow runs _and_ pushing to a standard binary cache outside of the Actions environment.
These aren't major drawbacks given the limited scope of the Action but they are worth keeping in mind.
## Kudos and thanks
The Magic Nix Cache is a collaborative project with [Zhaofeng Li][zhaofeng].
Zhaofeng is a major contributor to the Nix community perhaps best known as the author of two widely used projects:
- [Attic], a self-hosted, multi-tenant, [S3]-compatible Nix [binary cache][binary]
- [Colmena], a [NixOps]-inspired deployment tool for [NixOS] written in [Rust]
We'd like to express our deep gratitude to Zhaofeng for his tremendous work on this project.
The Magic Nix Cache is part of an already-impressive legacy and we're beyond eager to see what he contributes to Nix in the coming years.
## Implications
The Magic Nix Cache provides immediate benefits to your Nix workflows in GitHub Actions with few trade-offs or downsides.
And while it isn't ideal [for _all_ use cases](#drawbacks), we're confident that a wide variety of Nix projects can reduce Nix-related build times in Actions by 30-50% (and in some cases even more) with no configuration changes beyond the [single line of YAML](#one-liner) shown above.
As with everything we build at [Determinate Systems][detsys], we hope that this will provide a dramatically improved experience around using Nix.
Our internal usage of Nix in GitHub Actions has already been radically improved by the Magic Nix Cache, and we're confident that your experience with it will mirror our own.
[action]: https://github.com/DeterminateSystems/magic-nix-cache-action
[actions]: https://github.com/features/actions
[actions-cache]: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows
[attic]: https://github.com/zhaofengli/attic
[binary]: https://zero-to-nix.com/concepts/caching/#binary-caches
[cache-api]: https://docs.github.com/en/rest/actions/cache
[caching]: https://zero-to-nix.com/concepts/caching
[cachix]: https://cachix.org
[colmena]: https://github.com/zhaofengli/colmena
[derivation]: https://zero-to-nix.com/concepts/derivations
[detsys]: /
[dev]: /blog/nix-github-actions
[dev-env]: https://zero-to-nix.com/concepts/dev-env
[docker]: https://docker.com
[features]: https://zero-to-nix.com/#features
[hash]: https://zero-to-nix.com/concepts/nix-store#store-paths
[magic-nix-cache]: https://github.com/DeterminateSystems/magic-nix-cache
[minio]: https://github.com/minio/minio
[nix]: https://zero-to-nix.com
[nixos]: https://zero-to-nix.com/concepts/nixos
[nixops]: https://github.com/NixOS/nixops
[nixpkgs]: https://zero-to-nix.com/concepts/nixpkgs
[realises]: https://zero-to-nix.com/concepts/realisation
[s3]: https://aws.amazon.com/s3
[rust]: https://rust-lang.org
[security]: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache
[store]: https://zero-to-nix.com/concepts/nix-store
[zhaofeng]: https://github.com/zhaofengli
---
**Introducing the Nix Flake Checker**
Published: June 20, 2023
URL: https://determinate.systems/blog/flake-checker
Quite possibly the best thing about the [Nix] ecosystem is that there's a small army of people hard at work improving [Nixpkgs], the largest software package repository in existence and one of the most active repos on [GitHub], every single day.
Not only are they constantly adding brand new packages for stuff that you might want to use—over 80,000 packages and counting!—they're also updating existing packages, which sometimes even includes fixes for critical security vulnerabilities.
But to take full advantage of this steady drumbeat of progress, it's important that you follow some best practices.
To help you adopt those practices, we at [Determinate Systems][detsys] have created a tool called [Nix Flake Checker][flake-checker] and we're excited to release it to the Nix community.
## What Nix Flake Checker looks for
In any [flake-enabled][flakes] Nix project, Nix Flake Checker runs three checks on the root-level [Nixpkgs] dependencies in your [flake inputs][inputs]:
1. If your Nixpkgs input uses a specific Git branch, such as `nixpkgs-unstable`, it needs to be a supported release branch.
Nixpkgs' release branches stop receiving updates roughly 7 months after release and then gradually become more and more out of date—read: potentially insecure—over time.
Release branches are also certain to have good [binary cache][cache] coverage, which other branches can't promise.
1. Your Nixpkgs input must have been updated in the last 30 days.
Because Nixpkgs sees a steady stream of community updates, the "older" your Nixpkgs revision, the less likely you'll be to benefit from these updates.
30 days is somewhat arbitrarily chosen and potentially too lax; we may make this more strict in the future.
1. If your Nixpkgs input is a GitHub repo, it needs to have the [`NixOS`][nixos-org] org as the owner.
In principle, you could have a Nixpkgs input owned by someone else, for example `inputs.nixpkgs.url = "github:EvilCo/nixpkgs"`.
But we don't recommend this, first because forks and other non-upstream variants of Nixpkgs can introduce security vulnerabilities and unexpected behaviors, and second because those forks are often not kept up to date (though maybe they would be if they used Nix Flake Checker :wink:).
Upstream Nixpkgs isn't bulletproof—nothing in software is!—but it has a wide range of security measures in place, most notably continuous integration testing with [Hydra](https://hydra.nixos.org), that mitigate a great deal of supply chain risk.
It's important to keep in mind that Nix Flake Checker checks _all_ root-level Nixpkgs inputs.
In a flake input group like this, three inputs would be checked:
```nix
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs";
nixpkgs-unstable.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
nixpkgs-very-specific-ref.url = "github:NixOS/nixpkgs/0b23874b968c333abd4434701c2bbd552da8af8b";
};
```
It does _not_, however, check non-root Nixpkgs inputs for other inputs.
If your flake uses [`rust-overlay`][rust-overlay] as an input, for example, then Nix Flake Checker doesn't check `rust-overlay`'s own [Nixpkgs input][rust-overlay-nixpkgs], even though that input is registered in your `flake.lock`.
If you used a `follows` statement to pin `rust-overlay`'s Nixpkgs to your root Nixpkgs, however, then Nix Flake Checker _would_ cover that:
```nix
{
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable"; # Root Nixpkgs
inputs.rust-overlay = {
url = "github:oxalica/rust-overlay";
inputs.nixpkgs.follows = "nixpkgs"; # rust-overlay pin to root Nixpkgs
};
}
```
## Run Nix Flake Checker locally
Run this command in the same directory as the lock file to check the `flake.lock` in a Nix project:
```shell
nix run "github:DeterminateSystems/flake-checker"
# Or specify a different location
nix run "github:DeterminateSystems/flake-checker" -- /some/other/project/flake.lock
```
This provides a plaintext assessment of your `flake.lock`'s health, indicating any issues with your Nixpkgs inputs and providing pointers on how to bring your flake in line with best practices.
You can also add it to your Nix profile:
```shell
nix profile install "github:DeterminateSystems/flake-checker"
```
And of course you can add it to a Nix [development environment][dev].
## Automatic checks using the Nix Flake Checker Action \{#action}
While running ad-hoc checks locally and fixing issues is nice, we recommend using automated approaches whenever possible.
For [GitHub Actions][actions] users, we've created the [Determinate Flake Checker Action][action].
To add the checker to your pipeline:
```yaml
- name: Check Nix flake inputs
uses: DeterminateSystems/flake-checker-action@v4
```
Here's an example of using it in a real pipeline:
```yaml
jobs:
nix_build:
name: Build Nix targets
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Check Nix flake inputs
uses: DeterminateSystems/flake-checker-action@main
- name: Install Nix
uses: DeterminateSystems/determinate-nix-action@v3
- name: Build Nix package
run: nix build
```
The Action outputs a [job summary][summary] as [Markdown].
If your `flake.lock` has a clean bill of health, you should see a summary like this:
But if your `flake.lock` has issues, you'll see something like this:
As you can see, the summary points out not only _what_ is wrong but also how you can fix it.
In this case, if you click on **What to do** the summary tells you to use one of a select set of officially supported branches and then shows you how to update your flake inputs accordingly.
For the sake of your flakes' long-term health, we recommend using the Nix Flake Checker Action in conjunction with our [update-flake-lock] Action, which automatically submits [`nix flake update`][update] pull requests to flake-backed Nix projects on GitHub.
## Telemetry
We'd like you to be aware that Nix Flake Checker collects a bit of anonymized, aggregated telemetry that [Determinate Systems][detsys] uses to measure the impact of our efforts.
You can see a full breakdown of what information it collects in the [Nix Flake Checker README][telemetry].
## Implications
At [Determinate Systems][detsys], our goal is to [make Nix better][better].
[Pinning] dependencies is quite possibly _the_ killer feature of [Nix flakes][flakes], and flakes become all the more powerful when they're kept up to date with the most recent work of the incredible Nix community.
We believe that supporting best practices in flake dependencies is a small but substantial step forward for developers and orgs that use Nix.
We're open to feedback on both [Nix Flake Checker][flake-checker] and the [Nix Flake Checker Action][action], so do feel free to submit issues and pull requests.
We welcome bug fixes, feature requests, and everything between.
[action]: https://github.com/DeterminateSystems/flake-checker-action
[actions]: https://github.com/features/actions
[better]: /blog/we-want-to-make-nix-better
[cache]: https://zero-to-nix.com/concepts/caching#binary-caches
[detsys]: https://determinate.systems
[dev]: https://zero-to-nix.com/concepts/dev-env
[flake-checker]: https://github.com/DeterminateSystems/flake-checker
[flakes]: https://zero-to-nix.com/concepts/flakes
[github]: https://github.com
[inputs]: https://zero-to-nix.com/concepts/flakes#inputs
[markdown]: https://markdownguide.org
[nix]: https://nixos.org
[nixos-org]: https://github.com/NixOS
[nixpkgs]: https://github.com/NixOS/nixpkgs
[pinning]: https://zero-to-nix.com/concepts/pinning
[rust-overlay]: https://github.com/oxalica/rust-overlay
[rust-overlay-nixpkgs]: https://github.com/oxalica/rust-overlay/blob/e75da5cfc7da874401decaa88f4ccb3b4d64d20d/flake.nix#L8
[summary]: https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries
[telemetry]: https://github.com/DeterminateSystems/flake-checker#telemetry
[update]: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-update.html
[update-flake-lock]: https://github.com/DeterminateSystems/update-flake-lock
---
**Declarative GNOME configuration with NixOS**
Published: June 1, 2023
URL: https://determinate.systems/blog/declarative-gnome-configuration-with-nixos
I _adore_ tinkering with my machine, trying new tools, extensions, themes, and ideas. When I was younger, it was just a way to learn. Now, it's a way for me to refine my workspace and bring myself small joys.
While tinkering can be fun, it can be a chore to set up a new machine, keep configurations up to date between machines, or even just remember to keep up to date backups.
What about when we want to configure a whole desktop environment? While [NixOS](https://nixos.org/) offers configuration settings like `services.gnome.gnome-keyring.enable` for system-wide features, there's a lack of knobs when you want to set things like user-specific GNOME 'Favorite Apps' or extensions.
Let's explore a useful addition to your NixOS configuration: [Home Manager](https://nix-community.github.io/home-manager/) and its [`dconf`](https://wiki.gnome.org/Projects/dconf) module.
> This article uses [Nix flakes](https://nixos.wiki/wiki/Flakes) which is an experimental feature. You may need to set this in your configuration:
>
> ```nix showLineNumbers
> nix.settings.experimental-features = [ "flakes" "nix-command" ];
> ```
## Getting Home Manager set up
Home manager a tool from the Nix ecosystem that helps you take the declarative ideals of Nix/NixOS and apply them to your user's home directory (`$HOME`). It plugs in (as a [NixOS module](https://nixos.wiki/wiki/NixOS_modules)) into an existing NixOS configuration, or can be installed on different Linux as a user service.
In order to make some parts of your configuration declarative, Home Manager might take control of certain file paths, or set various options in things like `dconf`.
Because of its job, Home Manager can make updating your configuration **feel** more error-prone, but don't fear: Use a VCS like [`git`](https://git-scm.com/) to store your configuration. If your Home Manager setup breaks or acts strange for any reason, check the service status via `systemctl status home-manager-$USER` and `journalctl -u home-manager-$USER.service`. When in doubt, roll back your configuration and delete any files the errors are complaining about.
> In our example, the user is named `ana`, and the machine is named `gizmo`.
In your Nix flake, add the input for Home Manager and ensure it follows the `nixpkgs` you're using:
```nix showLineNumbers
# flake.nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
home-manager = {
url = "github:nix-community/home-manager";
inputs.nixpkgs.follows = "nixpkgs";
};
# ...
};
# ...
}
```
If you don't already have GNOME configured, you can do that via a `nixosModule` like this:
```nix showLineNumbers
# flake.nix
{
# ...
outputs = { self, nixpkgs, home-manager }:
let
# ...
in {
# ...
nixosModules = {
# ...
gnome = { pkgs, ... }: {
config = {
services.xserver.enable = true;
services.xserver.displayManager.gdm.enable = true;
services.xserver.desktopManager.gnome.enable = true;
environment.gnome.excludePackages = (with pkgs; [
gnome-photos
gnome-tour
]) ++ (with pkgs.gnome; [
cheese # webcam tool
gnome-music
gedit # text editor
epiphany # web browser
geary # email reader
gnome-characters
tali # poker game
iagno # go game
hitori # sudoku game
atomix # puzzle game
yelp # Help view
gnome-contacts
gnome-initial-setup
]);
programs.dconf.enable = true;
environment.systemPackages = with pkgs; [
gnome.gnome-tweaks
]
};
};
};
};
}
```
Next, add a `nixosModule` that enables `home-manager`:
```nix showLineNumbers
# flake.nix
{
# ...
outputs = { self, nixpkgs, home-manager }:
let
# ...
in {
# ...
nixosModules = {
# ...
declarativeHome = { ... }: {
config = {
home-manager.useGlobalPkgs = true;
home-manager.useUserPackages = true;
};
};
};
};
}
```
I create a `nixosModule` for each of my users (you may have another way, feel free to do that):
```nix showLineNumbers
# flake.nix
{
# ...
outputs = { self, nixpkgs, home-manager }:
let
# ...
in {
# ...
nixosModules = {
# ...
users-ana = ./users/ana;
};
};
}
```
You can enable Home Manager for your user like this:
```nix showLineNumbers
# users/ana/default.nix
{ ... }:
{
config = {
home-manager.users.ana = ./home.nix;
users.users.ana = {
# ...
};
};
}
```
Now create the `home.nix` referenced above:
```nix showLineNumbers
# users/ana/home.nix
{ ... }:
{
home.username = "ana";
home.homeDirectory = "/home/ana";
# ...
programs.home-manager.enable = true;
home.stateVersion = "22.05";
}
```
Before enabling, ensure your `nixosConfiguration` has these modules, as well as `home-manager.nixosModules.home-manager`:
```nix showLineNumbers
# flake.nix
{
# ...
outputs = { self, nixpkgs, home-manager }:
let
# ...
in {
# ...
nixosConfigurations = {
gizmo = {
system = "aarch64-linux";
modules = with self.nixosModules; [
({ config = { nix.registry.nixpkgs.flake = nixpkgs; }; })
# ...
home-manager.nixosModules.home-manager
gnome
declarativeHome
users-ana
];
};
# ...
}
};
}
```
With that, you should be able to switch into the new configuration:
```bash
nixos-rebuild switch --flake .#gizmo
```
Validate it worked by reviewing the output:
```bash
$ systemctl status home-manager-ana.service
● home-manager-ana.service - Home Manager environment for ana
Loaded: loaded (/etc/systemd/system/home-manager-ana.service; enabled; preset: enabled)
Active: active (exited) since Mon 2022-09-26 22:15:32 PDT; 2s ago
Process: 54958 ExecStart=/nix/store/nbhk58wgzvm2w8npi18qzjnn0xjcs3aw-hm-setup-env /nix/store/ig2vhy0pa4rvlkkdc511vyb6plp89x5a-home-manager-generation (code=exited, status=0/SU>
Main PID: 54958 (code=exited, status=0/SUCCESS)
IP: 0B in, 0B out
CPU: 377ms
```
If you see errors, dig deeper via ` journalctl -u home-manager-ana.service`.
## Declaratively configuring GNOME
There are a lot of knobs to set in GNOME.
GNOME breaks down into having GTK3/4 (which has UI, icon, and cursor themes), as well as an group of fairly tightly integrated components which are primarily configured by `dconf`, which most folks configure via `gnome-settings` or the settings panels of the relevant applications.
If you're curious and wanted to watch how/if various `dconf` settings get changed when doing things, you can 'watch' while you click around an application:
```bash
$ dconf watch /
/org/gnome/control-center/last-panel
'network'
/system/proxy/mode
'none'
/org/gnome/control-center/last-panel
'background'
/org/gnome/desktop/interface/color-scheme
'default'
/org/gnome/desktop/interface/color-scheme
'prefer-dark'
```
If it's too noisy, you can limit what you see by changing the `/` to a selector, for example `/org/gnome/desktop/`.
Home Manager offers a `dconf` module, which we can use to declaratively set these values.
## GTK3/GTK4 cursor, icon, and window themes
To set the GTK icon theme, first search for a theme. [this search](https://search.nixos.org/packages?channel=unstable&from=0&size=200&sort=relevance&type=packages&query=-gtk-theme) or [this one](https://search.nixos.org/packages?channel=unstable&from=0&size=200&sort=relevance&type=packages&query=-theme) can help you find already packaged themes. You can click the "Source" button on any of those packages to see the expression used to package it, just in case you end up needing to make your own.
Here are the relevant settings, with some examples of what I've found that I like:
```nix showLineNumbers
# users/ana/home.nix
{ pkgs, ... }:
{
# ...
gtk = {
enable = true;
iconTheme = {
name = "Papirus-Dark";
package = pkgs.papirus-icon-theme;
};
theme = {
name = "palenight";
package = pkgs.palenight-theme;
};
cursorTheme = {
name = "Numix-Cursor";
package = pkgs.numix-cursor-theme;
};
gtk3.extraConfig = {
Settings = ''
gtk-application-prefer-dark-theme=1
'';
};
gtk4.extraConfig = {
Settings = ''
gtk-application-prefer-dark-theme=1
'';
};
};
home.sessionVariables.GTK_THEME = "palenight";
# ...
}
```
> Want to set the **GNOME Shell theme?** We do this [below](#gnome-extensions) after discussing `dconf` a bit more.
Finding the `name` field for a given package can be a bit inconsistent. 😔
Most of the time, you can guess it, or copy it from what shows up when you check with `gnome-tweaks`. Usually, you can find the package repository via the "Homepage" link on the searches listed above, or in the expression via the `meta.homepage` ([example](https://github.com/NixOS/nixpkgs/blob/62228ccc672ed000f35b1e5c82e4183e46767e52/pkgs/data/themes/gtk-theme-framework/default.nix#L32)) or `src` fields ([example](https://github.com/NixOS/nixpkgs/blob/62228ccc672ed000f35b1e5c82e4183e46767e52/pkgs/data/themes/gtk-theme-framework/default.nix#L7-L9)). From there you can usually find some listing of the theme name ([example](https://github.com/jnsh/arc-theme/blob/b9b98cb394f8acf0a09f3601e3294d406b8e40a5/meson.build#L13-L18)).
**To use a theme not already packaged**, you'll need to take a [good starting point](https://github.com/NixOS/nixpkgs/blob/62228ccc672ed000f35b1e5c82e4183e46767e52/pkgs/data/themes/arc/default.nix), edit it, then add your custom package to your flake:
```nix showLineNumbers
# flake.nix
{
# ...
outputs = { self, nixpkgs, home-manager }:
let
supportedSystems = [ "x86_64-linux" "aarch64-linux" ];
forAllSystems = f: nixpkgs.lib.genAttrs supportedSystems (system: f system);
# ...
in {
# ...
overlays.default = final: prev: {
my-artistanal-theme = final.callPackage ./packages/my-artisanal-theme { };
# ...
};
packages = forAllSystems
(system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [ self.overlays.default ];
# ...
};
in
{
inherit (pkgs) my-artisanal-theme;
};
}
```
Once done, you should be able to use it by setting, for example, `gtk.theme.package = pkgs.my-artisanal-theme`.
## Setting GNOME options
As mentioned [above](#declaratively-configuring-gnome), most GNOME settings exist in `dconf`. Run `dconf watch /` and set whatever option you're looking to declaratively persist, and observe the output:
Here's what I see when I run `gnome-settings` and visit the 'Appearance' pane, then in 'Style' click between 'Light' and 'Dark'.
```bash
$ dconf watch /
# ...
/org/gnome/desktop/interface/color-scheme
'default'
/org/gnome/desktop/interface/color-scheme
'prefer-dark'
```
Let's try setting those from the command line an observing the `gnome-settings` window change:
```bash
dconf write /org/gnome/desktop/interface/color-scheme "'default'"
# Observe `gnome-settings` being light
dconf write /org/gnome/desktop/interface/color-scheme "'prefer-dark'"
# Observe `gnome-settings` being dark
```
Using this information, we can add this to the user configuration:
```nix showLineNumbers
# users/ana/home.nix
{ pkgs, ... }:
{
# ...
# Use `dconf watch /` to track stateful changes you are doing, then set them here.
dconf.settings = {
"org/gnome/desktop/interface" = {
color-scheme = "prefer-dark";
};
};
}
```
After a bit of tweaking, you might end up with something like this:
```nix showLineNumbers
# users/ana/home.nix
{ pkgs, ... }:
{
# ...
dconf.settings = {
# ...
"org/gnome/shell" = {
favorite-apps = [
"firefox.desktop"
"code.desktop"
"org.gnome.Terminal.desktop"
"spotify.desktop"
"virt-manager.desktop"
"org.gnome.Nautilus.desktop"
];
};
"org/gnome/desktop/interface" = {
color-scheme = "prefer-dark";
enable-hot-corners = false;
};
"org/gnome/desktop/wm/preferences" = {
workspace-names = [ "Main" ];
};
"org/gnome/desktop/background" = {
picture-uri = "file:///run/current-system/sw/share/backgrounds/gnome/vnc-l.png";
picture-uri-dark = "file:///run/current-system/sw/share/backgrounds/gnome/vnc-d.png";
};
"org/gnome/desktop/screensaver" = {
picture-uri = "file:///run/current-system/sw/share/backgrounds/gnome/vnc-d.png";
primary-color = "#3465a4";
secondary-color = "#000000";
};
};
}
```
## GNOME Extensions
Once user extensions are enabled, extensions can be added to the `home.packages` set then enabled in `dconf.settings."org/gnome/shell".enabled-extensions`. After, they can be configured just as any other GNOME option as described [just above](#setting-gnome-options).
```nix showLineNumbers
# users/ana/home.nix
{ pkgs, ... }:
{
# ...
dconf.settings = {
# ...
"org/gnome/shell" = {
disable-user-extensions = false;
# `gnome-extensions list` for a list
enabled-extensions = [
"user-theme@gnome-shell-extensions.gcampax.github.com"
"trayIconsReloaded@selfmade.pl"
"Vitals@CoreCoding.com"
"dash-to-panel@jderose9.github.com"
"space-bar@luchrioh"
];
};
};
home.packages = with pkgs; [
# ...
gnomeExtensions.user-themes
gnomeExtensions.tray-icons-reloaded
gnomeExtensions.vitals
gnomeExtensions.dash-to-panel
gnomeExtensions.sound-output-device-chooser
gnomeExtensions.space-bar
];
}
```
A GNOME Shell theme can be picked like this:
```nix showLineNumbers
# users/ana/home.nix
{ pkgs, ... }:
{
# ...
dconf.settings = {
# ...
"org/gnome/shell" = {
disable-user-extensions = false;
enabled-extensions = [
"user-theme@gnome-shell-extensions.gcampax.github.com"
];
"org/gnome/shell/extensions/user-theme" = {
name = "palenight";
};
};
};
home.packages = with pkgs; [
# ...
gnomeExtensions.user-themes
palenight-theme
];
}
```
## Troubleshooting
> I made a bunch of changes then ran `nixos-rebuild switch`, but some things didn't change?
Log out and log back in. I found some things just didn't set themselves until you did!
> I accidentally altered GNOME settings which I'd set via Home Manager, and they aren't changing back on a `nixos-rebuild switch`?
It is possible to change `dconf` values once set via Home Manager, if this happens and `nixos-rebuild switch` isn't causing a change, **you may need to restart the `home-manager` service** with `systemctl restart home-manager-$USER`. If that fails, make a change (for example `touch`) your user home configuration first.
> Some changes I made are not reflecting when I `nixos-rebuild switch`?
Check `systemctl status home-manager-$USER` and ensure the service started successfully, if not, dig in with `journalctl -u home-manager-$USER` and make sure to carefully read the error.
> The setting I want isn't tracked by Home Manager or `dconf`?
It might be complicated. You can try seeing if the program creates an entry in your XDG directories, such as `~/.config` or `~/.cache`. If so, you can often provision content into the file with the [`home.file`](https://rycee.gitlab.io/home-manager/options.html#opt-home.file) option. Please note this will make the file unwritable, which may impact some programs.
## Conclusion
Once various all of your preferred settings are persisted, it becomes straightforward to share key settings and extensions between multiple machines, or even architectures. By taking a little bit more time when configuring our system, we can avoid having to do it again.
Using these strategies I was able to have 3 different machines (an x86_64 desktop workstation, an x86_64 laptop, and a headed aarch64 server) with the same settings, and keep them all consistent so I can spend more time doing what I enjoy (tinkering) instead of what I don't (re-configuring machines).
If you're looking for a complete home configuration, you can check out mine [here](https://github.com/Hoverbear-Consulting/flake/blob/89cbf802a0be072108a57421e329f6f013e335a6/users/ana/home.nix).
Here's what it looks like:
---
**Packaging Open Policy Agent policies with Nix**
Published: May 22, 2023
URL: https://determinate.systems/blog/open-policy-agent
[Open Policy Agent][opa] (**OPA**) is an open source, general-purpose [policy engine][engine] that enables you to express policy logic in a [Datalog]-flavored DSL called [Rego] and evaluate those policies against [JSON].
Policy is of vital importance in many domains of software—complex authorization is probably the most well-known use case—but policy logic can be quite laborious to write in standard programming languages.
OPA and Rego provide an elegant alternative to the nested `if`/`else` spaghetti code that we're all painfully familiar with.
I've [written about][blog-post], [presented on][presentation], and [contributed to][prs] OPA in the past, so my interest isn't new—in fact, it's been piqued again and again over the years and I'm always looking for excuses to do new things with it.
The three things I like most about OPA:
- Because it "speaks" JSON, you can use it to evaluate policies in any domain that also speaks JSON, and that now includes just about every domain of software, from [security] to [deployment] to [networking] to [messaging] and beyond.
- You can run it in a variety of ways: on the command line using the [`opa`][opa-cli], by POSTing JSON to OPA's [REST API][api], by using it as a [library] (in a [Go] program), and, most promisingly for my use case here, by compiling it to [WebAssembly][wasm] (Wasm) and running it in one of the many places Wasm now runs (imagine evaluating policies using edge workers or in the browser).
- Beyond its [concision], [Rego] has a robust [standard library][stdlib], which includes functions for things like [bitwise operations][bitwise], [globs], [regular expressions][regex], [cryptography][crypto], and much more.
So with OPA you get both broad applicability and a highly flexible usage model, which to me is quite a formidable pairing.
## An unsatisfied use case
By far the most straightforward way to use OPA is with the [OPA CLI][cli].
This command, for example, evaluates `user-info.json` against the [Rego] policy in `rbac-policy.rego`:
```shell
opa eval \
--data rbac-policy.rego \
--input user-info.json
```
One drawback of this approach is that OPA needs to be installed in your environment and to have access to both the JSON to be evaluated and the Rego policy.
This can make bundling and distribution rather non-trivial in places like continuous integration environments.
What I'd like instead is to provide ready-made CLI tools that I can use to evaluate policies without OPA installed or keeping the Rego files locally.
In other words, I'd like to fetch a purpose-built binary for a specific policy and provide it with JSON to evaluate, like this:
```shell
rbac-evaluate --input-path user-info.json
```
In this post, I'll show you how I used [Nix] to accomplish precisely that—plus some [Rust] in places where I needed a standard programming language to provide the CLI logic.
If you want to take a look now, check out the [nix-policy][gh] project on Determinate Systems' [GitHub org][org].
## What I built
nix-policy takes [Rego] policies and uses Nix and Rust to convert them into standalone CLI tools.
Let's say you have a Rego policy named `k8s_admission.rego` that provides [admission control][k8s] for a Kubernetes API.
You could use nix-policy to generate a CLI tool called `k8s-admission` that provides this help output when you run `k8s-admission --help`:
```
A policy evaluator for the k8s_admission.rego Kubernetes admission control policy
Usage: k8s-admission [OPTIONS]
Options:
-d, --data Policy data object
-d, --data-path Path to policy data JSON object
-i, --input Policy input object
-i, --input-path Path to policy input JSON object
-o, --output