Improved evaluation times with pre-resolved Nix store paths

Building things with Nix is pretty amazing. You point Nix at a flake reference for a derivation and say “build!” It then spins up a sandbox and reproducibly creates real bytes on disk. Those bytes on disk could be a single hello.txt file or a hefty Docker image or a fully functional NixOS system—or much else besides.

But the power of nix build comes at a cost, and not just in terms of disk space. Before it writes anything to disk, Nix first needs to evaluate the store path of the derivation that it’s going to realise. It then uses that store path to determine whether it can pull the path from a cache like FlakeHub Cache or whether it needs to build it instead. Finally, it applies this logic recursively throughout the entire dependency tree of the derivation and you end up with the desired bytes on disk.

The evaluation tax

Even if you can (ideally) pull a path from a cache instead of building it, you still have to pay Nix’s evaluation tax in determining the derivation’s store path in the first place. This cost is no big deal if you’re on a modern workstation or a beefy build server. But on resource-constrained devices, such as smart sensors, Raspberry Pis, or industrial controllers, even pulling from a cache can use a ton of memory if you need to evaluate the store path—something you’d surely like to avoid at all costs.

The solution: resolved store paths

Fortunately, FlakeHub now offers an elegant solution to this problem: resolved store paths. With this feature, you can do two things that have not yet been possible in the Nix ecosystem:

1. You can provide FlakeHub with a flake output reference and get the store path for that reference without using Nix. With the store path in hand, you can pull directly from FlakeHub Cache with no evaluation tax.
2. You can apply a NixOS, Home Manager, or nix-darwin configuration to the current host solely with a flake reference—again pulled directly from FlakeHub Cache without incurring an evaluation tax.

Working with resolved store paths

You can work with resolved store paths using fh, the CLI for FlakeHub, which enables you to fetch store paths and apply configurations from flake references.

fh resolve

fh, the CLI for FlakeHub, has a resolve command that resolves flake references to store paths by fetching them directly from FlakeHub. Here’s an example:

fh resolve \
  "DeterminateSystems/store-paths/*#nixosConfigurations.baseline"

It also works with the full flake reference:

fh resolve \
  "https://flakehub.com/f/DeterminateSystems/store-paths/*#nixosConfigurations.baseline"

Both of these fh commands return this store path immediately:

/nix/store/84g3x33v5srlljhfaskwy7mmar5697vn-nixos-system-minimal-24.05.20240714.53e81e7

It may not be clear why this is special, as you can get the same result using plain old nix eval:

nix eval --raw "https://flakehub.com/f/DeterminateSystems/garage-door/0.1.9#nixosConfigurations.turner.config.system.build.toplevel"

But this command requires Nix to be available and takes several seconds and a fair amount of memory and disk space to complete. With fh resolve, by contrast, fh fetches an already calculated store path stored in FlakeHub’s database. In a variety of deployment scenarios, using fh is a vastly superior option.

fh apply

fh, the CLI for FlakeHub, has an apply command that you can use to apply NixOS, Home Manager, and nix-darwin configurations to the current system. fh apply is powered by the same path resolution feature mentioned above, meaning that you can use it to, for example, apply a NixOS configuration to the current host.

NixOS

Let’s say that you have a NixOS configuration as a flake output on your my-org/nixos-configs repo, while that configuration is being pushed to FlakeHub Cache. With fh, you could apply that configuration using this single command:

fh apply nixos "my-org/nixos-configs/0.1#nixosConfigurations.staging-server"

If you leave off the flake output reference—nixosConfigurations.staging-server in this case—fh “magically” infers that you want the nixosConfigurations.$(hostname) output. If that’s the case, you can shorten the reference to this:

fh apply nixos "my-org/nixos-configs/0.1"

NixOS on Amazon Web Services (AWS)

The value of this new approach becomes clear in this little snippet of code:

determinate-nixd login aws
fh apply nixos "my-org/nixos-configs/0.1"

If you have an EC2 instance with Determinate Nix installed, this is all you need to run to update your NixOS system configuration. Determinate Nixd logs you in, which enables you to use private flakes, and then fh does its thing. As before, it does so without needing to evaluate the store path, which makes this speedier than your typical NixOS deployment.

And because FlakeHub supports semantic versioning (SemVer) for flakes, the fh apply operation would automatically use the most recent release within the supplied version constraint, so you could also use constraints like * (whatever is latest), =0.1.2 (exact match), or ~1.2 (flexible patch).

Home Manager

While fh apply for NixOS works swimmingly in cloud environments like AWS, it also supports applying Home Manager configurations. Here’s an example:

fh apply home-manager "my-org/home-configs/0.1#homeConfigurations.justme"

fh apply has a default flake output name of homeConfigurations.$(whoami), so this command without the flake output would work if the output name conformed to that:

fh apply home-manager "my-org/home-configs/0.1"

nix-darwin

And finally, fh apply also supports nix-darwin. We at Determinate Systems are committed to providing first-class Nix support on macOS and this is a testament to that. Here’s an example command:

fh apply nix-darwin "my-org/macos-configs/0.1#darwinConfigurations.justme-aarch64-darwin.system"

The default flake output name for nix-darwin is darwinConfigurations.${hostname}, where hostname is the value of scutil --get LocalHostName. If your flake output name conforms to that, you can run this:

fh apply nix-darwin "my-org/macos-configs/0.1"

Generating store paths

Above, we saw how to use fh resolve and fh apply with resolved store paths. To be able to fetch those paths, however, you need to generate them as part of the flake publishing process. You can configure that with a one-liner in the flakehub-push Action:

- uses: DeterminateSystems/flakehub-push@main
  with:
    name: my-org/my-flake
    include-output-paths: true

With this configuration in place, the store paths for all of your flake’s outputs are evaluated and stored in FlakeHub’s database. As with everything on FlakeHub, those paths are available only to users with proper permissions. For public flakes, that’s anyone with fh installed; for private flakes that’s only those to whom you’ve explicitly granted access.

Here’s a full GitHub Actions workflow with resolved store paths enabled:

name: Publish flake with resolved store paths available

on:
  push:
    branches:
      - main

jobs:
  flakehub-publish:
    runs-on: ubuntu-22.04
    needs: build-docker-image
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: DeterminateSystems/nix-installer-action@main
      - uses: DeterminateSystems/flakehub-push@main
        with:
          name: omnicorp/web
          rolling: true # Publish on every push to the default branch
          visibility: private
          include-output-paths: true

With that output path published, users with access to the omnicorp/web flake could use fh to fetch a resolved path like this:

fh resolve "omnicorp/web/*#dockerImages.x86_64-linux.server"

Truly new horizons for Nix

FlakeHub now enables you to determine store paths without evaluating them, which unlocks some exciting new possibilities for Nix in the enterprise. With this feature, fh apply can apply Nix-based configurations for NixOS, Home Manager, and nix-darwin in a much more efficient way than is currently available in the Nix ecosystem. Beyond fh apply, we can imagine people using fh resolve for things like scripting or pulling artifacts like Docker containers from FlakeHub Cache (more on this soon).