Moving stuff around with Nix

Nix has an extremely broad feature set that enables you to do a lot of fun stuff. In past posts here on the Determinate Systems blog I’ve touted some of the more widely used features, such as reproducible development environments and declarative home environments, but today I want to focus on something less widely used: nix copy (or nix-copy-closure if you’re using the older, non-unified Nix CLI). This utility enables you to copy what are called Nix closures between machines, which can be quite handy in a variety of distribution and deployment scenarios.

Here’s an example nix copy command:

nix copy \
  --to ssh-ng://my-remote-host \
  "nixpkgs#apachakafka_3_3"

This command would copy, via SSH, everything required to run Apache Kafka v3.3 to a remote host with Nix installed. And as we’ll see in this post, the “everything required” is what sets nix copy apart from other tools.

Closures in Nix

In Nix, a package’s closure encapsulates everything needed to either build or run the package:

• Build closures include all the dependencies required to build the package, such as compilers, package managers, and shell utilities.
• Runtime closures include everything required to run any or all programs in the package, such as configuration files, dynamically linked libraries, or other programs (imagine a shell utility that relies on output from openssl or ffmpeg to work).

Which closure type you need to work with depends on the use case at hand, but runtime closures are, unsurprisingly, more common in deployment scenarios (as in the project I’ll talk about below).

By contrast, most tools that move stuff between machines, such as rsync and scp, typically “think” in terms of files and directories and are thus fundamentally procedural. If you need to copy a dependency tree to another machine you need to do so manually, which, as many of us have learned the hard way, is quite error prone. But nix copy thinks in terms of full closures, which are a fundamentally declarative abstraction. You tell Nix the “what” (the package to copy) and the entire dependency tree comes along with it. The “how” of this operation is baked into Nix, which pretty much can’t think in terms of plain old files and directories.

And because nix copy interacts with the Nix store, it can use the core mechanics of the Nix store and caching to make the copy operation efficient. On any given nix copy run you may need to build an entire closure from scratch or you may be able to get most of the closure from the target machine’s Nix store or a configured binary cache. If the target machine already has some dependencies in the Nix store, nix copy is essentially a no-op for those.

Why nix copy

There are many potential use cases for nix copy, but the one with the most clear value is maintaining a clean separation between machines that build packages and machines that use them. You can, for example, run a beefy CI machine that copies a Nix package that’s highly resource intensive to build onto less beefy machines that then use the package. In this example, a beefy CI machine copies a package called service-pkg defined in a local flake to a VM in the cloud:

# On the beefy CI machine
nix copy \
  --to ssh-ng://"${CLOUD_VM_IP}" \
  ".#packages.x86_64-linux.service-pkg"

Another option would be to stand up a cloud VM and copy a closure from a beefy CI machine:

nix copy \
  --from ssh-ng://"${CI_MACHINE_IP}" \
  ".#packages.x86_64-linux.service-pkg"

By default, Nix copies the runtime closure, but if for some reason we needed to copy the build closure we could apply the --derivation flag:

nix copy \
  --derivation \
  --to ssh-ng://"${CLOUD_VM_IP}" \
  ".#packages.x86_64-linux.service-pkg"

Example project

To demonstrate nix copy in action, I created a fairly straightforward example project:

https://github.com/DeterminateSystems/nix-copy-deploy

In this project, I use Terraform to stand up some Linux droplets on Digital Ocean. Once the droplets have been created, I run a shell script that reads the Terraform state file to gather a list of IPs for the droplets and then runs a few commands on each droplet via SSH:

• It installs Nix using the Determinate Nix Installer, an experimental tool that we at Determinate Systems [recently released]( https://determinate.systems/posts/determinate-nix-installer.
• It nix copy a Nix package defined in the local flakes onto the droplet (specifically a program called ponysay.
• It feeds the text Hello from nix copy! to ponysay, which then outputs a lovely greeting from a cartoon horse.

The command that accomplishes this:

nix copy \
  --to ssh-ng://root@"${DROPLET_IP}" \
  ".#packages.x86_64-linux.ponysay"

Standing up cloud VMs to run ponysay isn’t exactly a show-stopping use case, but I think it illustrates the mechanics of nix copy. You could replace ponysay here with any package output from any Nix flake: it could be a web service or Postgres or Kafka or even an entire NixOS configuration. If you can package it using Nix—and that’s pretty much anything imaginable—then you can nix copy it onto a machine with Nix installed and run it.

And unlike with many other deployment approaches, with nix copy you don’t have to fuss with rsync and shell scripts. You tell the Nix CLI what needs to be copied and Nix handles the how, no matter how intricate, of shuttling it to its destination.

Other features

S3 support

nix copy supports storage systems compatible with the S3 API. In this example, Nix copies a package closure defined in a local flake to a bucket in AWS Singapore:

nix copy \
  --to "s3://my-nix-stuff-bucket?region=ap-southeast-1" \
  ".#my-package"

Security

When you build or copy closures with Nix, you have the option of using substituters, which are binary caches that enable you to fetch already-built dependencies rather than building them anew. You can specify substituters (by URL) either per Nix CLI command invocation or in your Nix configuration. Binary caches can be public (available to all) or private. To use private caches, you need to supply public keys. The nix copy utility verifies that all paths are signed by these public keys when using substituters, which ensures that no one can push artifacts to your cache without a valid key. You can, however, explicitly disable public key checking with the —no-check-sigs flag. This doesn’t make private binary caches bulletproof, of course, but it does tighten up the supply chain.

Nix in multi-machine contexts

While Nix is fantastic for deterministic package builds, reproducible development environments, and much more, utilities like nix copy propel Nix beyond the “usual” capabilities we associate with package managers and unlock use cases suited for larger orgs and heterogeneous environments.

If you’ve found this post a little hum-drum, we’re okay with that. nix copy is kind of, well, boring. You give it something to copy, you specify a source and a destination, and that’s it. But there’s also a certain beauty to that—a beauty that may become more clear to you next time you’re debugging a mysteriously failed rsync run.