Changelog: a new upstream base, GC roots for nix eval, and a faster nix copy

We at Determinate Systems are pleased to announce the release of Determinate Nix version 3.22.2, based on upstream Nix 2.35.2. This release moves Determinate Nix onto a new upstream base and provides a number of other improvements:

• nix eval gains a --drv-link flag that creates a garbage collection (GC) root for derivations it instantiates.
• nix copy is now faster when the store on the other end runs an older daemon.
• Remote builds over multiplexed SSH no longer fail with a protocol mismatch when the control connection has gone stale.
• nix store optimise now skips missing store paths instead of failing.
• Determinate Nixd now knows how to authenticate to FlakeHub from Replit.
• Nix’s progress bar now behaves much better when you suspend, resume, or resize your terminal.

And as always, we included a pair of smaller improvements to round out the lineup.

A new upstream base: Nix 2.35

With the 3.22 series, Determinate Nix has moved onto upstream 2.35 and picked up a wide range of fixes. A few upstream highlights:

• Nix links mimalloc when it’s available, replacing the system allocator for non-GC allocations. Upstream has measured 5% to 12% reduction in wall clock time on evaluation workloads.
• Nix can now fetch from binary caches over HTTP/3 (QUIC), controlled by a new http3 setting that’s off by default. When you turn it on, Nix falls back to HTTP/2 or HTTP/1.1 for servers that don’t advertise QUIC.
• File transfer retries now use full jitter exponential backoff, treat HTTP 503 the same as 429, and honor the Retry-After response header. The delays are configurable through the new filetransfer-retry-delay, filetransfer-retry-max-delay, and filetransfer-retry-jitter settings.
• nix flake check now supports --print-out-paths and --out-link, and nix store delete supports --skip-alive for collecting garbage inside a closure.

nix eval can root the derivations it instantiates

Previously, when you ran nix eval --json or nix eval --raw to instantiate derivations, nothing protected the .drv files it writes from garbage collection. Now, nix eval takes a --drv-link flag that creates GC roots for the store paths in the result’s string context:

nix eval --raw --drv-link ./drv .#foo.drvPath

That gives you a ./drv symlink for the first path, ./drv-1 for the second, and so on. Unlike nix build --out-link, the symlinks point at .drv files rather than at build outputs, because nothing has been built yet. Passing --drv-link without --raw or --json is an error rather than a silent no-op.

nix copy is faster against older daemons

While nix copy runs, Nix registers temporary GC roots on the destination store so that a garbage collection operation on that host can’t delete the paths out from under the copy. Daemons new enough to register a batch of roots in a single call handle this fine. Older ones—from both older Determinate Nix and upstream Nix versions—have no batch operation, so Nix fell back to one round trip per path, which is painfully slow over a high-latency connection.

Nix no longer registers those roots at all when it’s talking to a daemon that can’t batch them. The trade-off is that garbage collection running on the remote store during a copy can delete the paths you just copied.

Remote builds survive a stale SSH control socket

Determinate Nix sets max-connections to 64, whereas upstream Nix defaults to 1, which means that Determinate Nix opens an SSH control connection and multiplexes your remote store connections through it by default. That’s faster, but it exposed a latent bug that the upstream default never reaches.

When the control connection exits, its socket goes stale, and the next command connection quietly falls back to a direct SSH. A direct connection runs LocalCommand=echo started, which Nix uses to keep progress bar output from garbling password prompts. Nix skips reading that started string when it thinks a control connection is handling the session, so the word leaked straight into the Nix protocol stream:

error: cannot open connection to remote store 'ssh-ng://build@10.10.127.43': protocol mismatch, got 'started'

Determinate Nix now overrides LocalCommand to a no-op on connections that go through the control socket, so there’s nothing left to leak whether the control connection is alive or not. Thanks to @DarthPJB for this fix, their first contribution to Determinate Nix.

nix store optimise skips missing paths

A path that’s registered as valid but missing from disk used to end the entire optimization pass:

error: getting status of "/nix/store/0mqak3v82iwqz1v3d7v1r6nwrbw4pwij-gn-fd3d768.drv": No such file or directory

A missing valid path is still a problem worth investigating, but it shouldn’t stop Nix from deduplicating everything else in your store. nix store optimise now warns about the path, skips it, and finishes the job.

A better-behaved progress bar

The progress bar now responds to the signals your terminal sends it:

• Hitting Ctrl-Z restores the cursor before Nix suspends, so you get your prompt back in a usable state.
• Resuming a suspended Nix redraws the progress bar instead of leaving you with a blank line.
• Resizing your terminal redraws the progress bar immediately at the new width.

Smaller improvements

• Fewer SQLite calls in the local store. Every path that Nix registers in the local store means a round trip through SQLite, and those round trips add up fast on a large instantiation. The local store now inserts and updates a row in a single statement, and caches path IDs instead of looking them up again and again. Instantiating one of our NixOS configurations on an empty store used to take 379,086 SQLite calls; it now takes 225,397, a reduction of roughly 40%. No configuration change is required.
• Leaner parallel evaluation. Parallel evaluation queues up work items for its worker threads, and until now every root value one of those items captured had to be wrapped in a std::shared_ptr so the item stayed copyable. Work items now own their values outright, which removes a layer of allocation and reference counting from parallel evaluation and from recursive operations over deep values, such as converting a large attribute set to JSON.

Determinate Nixd changes

This Determinate Nixd release comes with a new authentication method and a reliability fix.

Log in to FlakeHub from Replit. If you develop on Replit, Determinate Nixd can now authenticate to FlakeHub using Replit’s own identity, with no long-lived token to copy around:

determinate-nixd auth login replit

Pass --frn if you need to name your FlakeHub Resource Name (FRN) explicitly rather than taking the one Replit hands back.

A broken config file no longer stops the daemon. Previously, if Determinate Nixd couldn’t parse its JSON configuration file, it refused to start, which is a pretty rude awakening for something as trivial as a stray comma. Instead, it now warns you about the file it couldn’t read and carries on with its default configuration.

This release also updates the toolchain we build Determinate Nixd with, fixing some macOS build and linking issues along the way.

How to get Determinate Nix

If you already have Determinate Nix installed, you can upgrade to 3.22.2 with one Determinate Nixd command:

sudo determinate-nixd upgrade

If you don’t yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on macOS using our graphical installer:

Install Determinate Nix on macOS

With support for Apple Silicon (aarch64-darwin)

On Linux:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
  sh -s -- install --determinate

On NixOS, we recommend using our dedicated NixOS module or our NixOS ISO (NixOS installer for x86_64, NixOS installer for ARM) with Determinate Nix pre-installed.

On GitHub Actions:

on:
  pull_request:
  workflow_dispatch:
  push:
    branches:
      - main

jobs:
  nix-ci:
    runs-on: ubuntu-latest
    # Include this block to log in to FlakeHub and access private flakes
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@main
      - uses: DeterminateSystems/flake-checker-action@main
      - uses: DeterminateSystems/determinate-nix-action@v3
      - uses: DeterminateSystems/flakehub-cache-action@v3
      - run: nix flake check

In Amazon Web Services:

data "aws_ami" "detsys_nixos" {
  most_recent = true
  owners      = ["535002876703"] # Commercial
  # owners      = ["579351485434"] # GovCloud
  # owners      = ["129194717446"] # European Sovereign Cloud

  filter {
    name   = "name"
    values = ["determinate/nixos/epoch-1/*"]
  }

  filter {
    name   = "architecture"
    values = ["x86_64"]
  }
}