Introducing the Magic Nix Cache

We at Determinate Systems, along with our collaborator Zhaofeng Li, are thrilled to announce the release of the Magic Nix Cache, a GitHub Action that can dramatically speed up your Nix-related workflows. You can use it today—no signup, setup, or cost—by adding just one line to your YAML configuration:

yaml - uses: DeterminateSystems/magic-nix-cache-action@v2

Here’s a more complete example:

jobs:
  build:
    steps:
      - uses: actions/checkout@v3
      - name: Install Nix
        uses: DeterminateSystems/nix-installer-action@v4
      - name: Run the Magic Nix Cache
        uses: DeterminateSystems/magic-nix-cache-action@v2
      - name: Build server and client
        run: |
          nix build .#server
          nix build .#client

We’re confident that its power and ease of use will provide immediate benefits to the Nix community at large.

Why caching is important

We at Determinate Systems use Nix for all of our GitHub Actions builds. Synchronizing environments across dev, prod, and CI is, after all, one of Nix’s major strong points. But let’s be frank: Nix builds can be quite slow. Nix always realises a derivation’s entire dependency tree when it builds something, and that tree can be quite vast.

Fortunately, Nix is extremely good at caching. Because everything stored in the Nix store has a content-derived hash in the store path, Nix can quickly determine if something needs to be built or if it can be fetched from a known binary cache instead. Caching can cut build times by orders of magnitude (depending on the context).

To reap the benefits of caching in a CI environment like GitHub Actions, you previously needed to either:

Run your own binary cache backed by a storage platform like Amazon S3 or S3-compatible Minio.
Use a dedicated SaaS service like Cachix.

The Magic Nix Cache provides a new option that doesn’t require you to deploy any new infrastructure or sign up for an external service.

How it works

The Magic Nix Cache relies on a daemon called magic-nix-cache, which is written in Rust. This daemon acts just like a standard binary cache server from the standpoint of Nix, but with the important difference that it writes to the GitHub Actions Cache using the Github Actions Cache API instead of writing to disk, as a standard binary cache would.

When you add the Magic Nix Cache to an Actions workflow, here’s what happens:

When the Action is triggered, it first downloads and runs the binary for the magic-nix-cache daemon. The daemon then fetches a list of what’s currently stored in the GitHub Actions Cache.
The rest of your workflow logic, including all of your Nix commands, proceeds.
When the workflow is finished, the magic-nix-cache daemon calculates a list of Nix store paths that haven’t yet been written to the GitHub Actions Cache and writes them.

With this approach, the Magic Nix Cache caches everything written to the Nix store during your workflow run. That includes your Nix sources (including bulkier things like revisions of Nixpkgs), development environments, all the intermediate dependencies from nix build invocations, Docker images you build with Nix, and more. And it does this automatically, without requiring a workflow step of this sort:

# NOT necessary so don't do this ❌
- name: Cache Nix artifacts
  run: ./complex-nix-cache-script.sh

Security

The Magic Nix Cache provides the same security guarantees as the Actions Cache API, which includes cache isolation across different branches and tags. Concretely, this means that the cache for a project can never be polluted by a malicious pull request because workflow runs never restore caches created for child or sibling branches. If someone submits a pull request for a branch called uh-oh-bitcoin-miner-incoming, for example, the Magic Nix Cache will indeed cache the Nix artifacts for that workflow but nothing from that cache will ever be used by your other branches, including your default branch (such as main).

What about rate limits?

Rate limits are a persistent concern for GitHub Actions users, and many of us have been bitten by workflow runs that fail by inadvertently exceeding those limits. The Magic Nix Cache is subject to the rate limits of the Actions Cache API and large projects or builds may run up against those constraints. If that happens, you’ll see log messages like this:

error: unable to download 'http://127.0.0.1:37515/<...>': HTTP error 418
       response body:
       GitHub API error: API error (429 Too Many Requests): StructuredApiError { message: "Request was blocked due to exceeding usage of resource 'Count' in namespace ''." }

Fortunately, the Magic Nix Cache handles any rate limiting gracefully. If your workflow exceeds the limit when it’s pulling Nix dependencies from your Actions cache, Nix will build those dependencies rather than fetching them from the cache. And if your workflow exceeds the limit when pushing to the cache, the Magic Nix Cache will stop caching and wait for the next workflow run to push those store paths to the cache.

Rate limiting may thus increase build times on specific workflow runs but it will not cause your runs to fail.

Drawbacks

Because the Magic Nix Cache uses the Actions Cache API, the artifacts it caches are availably only to workflow runs for your project. This means that:

Caches are shared across workflow runs but not across GitHub repos. We suspect, however, that any potential speed benefits of cross-project caching would be marginal at best.
Caches aren’t publicly available. If you’d like your Actions workflows to be used by members of your org or Nix users more broadly, you’ll need to push your Nix artifacts to a standard binary cache. But there’s nothing preventing you from using the Magic Nix Cache to speed up future Nix workflow runs and pushing to a standard binary cache outside of the Actions environment.

These aren’t major drawbacks given the limited scope of the Action but they are worth keeping in mind.

Kudos and thanks

The Magic Nix Cache is a collaborative project with Zhaofeng Li. Zhaofeng is a major contributor to the Nix community perhaps best known as the author of two widely used projects:

Attic, a self-hosted, multi-tenant, S3-compatible Nix binary cache
Colmena, a NixOps-inspired deployment tool for NixOS written in Rust

We’d like to express our deep gratitude to Zhaofeng for his tremendous work on this project. The Magic Nix Cache is part of an already-impressive legacy and we’re beyond eager to see what he contributes to Nix in the coming years.

Implications

The Magic Nix Cache provides immediate benefits to your Nix workflows in GitHub Actions with few trade-offs or downsides. And while it isn’t ideal for all use cases, we’re confident that a wide variety of Nix projects can reduce Nix-related build times in Actions by 30-50% (and in some cases even more) with no configuration changes beyond the single line of YAML shown above.

As with everything we build at Determinate Systems, we hope that this will provide a dramatically improved experience around using Nix. Our internal usage of Nix in GitHub Actions has already been radically improved by the Magic Nix Cache, and we’re confident that your experience with it will mirror our own.