Nuenv: an experimental Nushell environment for Nix

I appreciate Bash for its stability and its near-ubiquity in Unix environments, but it has some clear drawbacks. I constantly get bitten by things like variable handling and string interpolation, I almost always need to supplement my Bash scripts with tools like awk, sed, curl, jq, and wget to get even basic things done, and I’d be hard-pressed to use “Bash” and “elegant” or “expressive” in the same sentence.

Despite shortcomings like these, Nixpkgs uses Bash as the builder in its standard environment (or stdenv), most notably in the stdenv.mkDerivation function. This wrapper over Nix’s built-in derivation function is used almost everywhere in Nixpkgs and acts as a de facto default in the Nix community.

I believe that standardizing on Bash as a known quantity was absolutely the right choice for the standard environment. But I’ve always been intrigued by the possibility of using a more powerful, ergonomically-minded shell to realise Nix derivations. So I decided to give it a try by creating a Nix realisation environment that uses Nushell, a powerful, cutting-edge shell written in Rust and under heavy development, instead of Bash. I’m calling it Nuenv and it’s proven to be quite a fruitful exercise and yielded some promising initial results.

Why I chose Nushell

There are many “alt” shells out there, like Fish, Elvish, and Oil. These are all exciting projects but Nushell really stands out to me. Here’s a non-exhaustive list of reasons why:

• It offers built-in support for JSON, CSV, YAML, TOML, SQLite, and even DataFrames.
• It offers a broad range of built-in commands that play nicely together and more or less replace tools like jq, curl, awk, cut, join, and sed.
• It enables you to create custom commands with typed inputs and auto-generated (and very pretty) help output.
• It supports nested data records, which gives it a kind of object-oriented flavor.
• It has built-in support for testing.
• It offers scoping mechanisms like modules and overlays.

While Nushell isn’t quite a general-purpose programming language, it certainly skirts the boundary between a shell and a full-fledged language. Not all of Nushell’s features are equally germane to an alternative environment for Nix, of course, but I think that these features show how ambitious the project is.

Here’s what I think Nushell has to offer a new Nix environment:

• It makes a clean separation between environment and standard variables. $foo would refer to a variable in the local scope but only $env.foo could refer to an environment variable. This enables you to avoid those un-fun variable name clashes that pop up so often in Bash.
• String handling is powerful enough to obviate the need for the usual smorgasbord of tools required for string parsing, like awk, sed, and tr.
• The custom commands
• with typed parameters that I mentioned above make the environment much more robust than Bash’s “stringly” typing and awkward function argument handling. Here’s an example custom command:

# Say hello to <name> and add a ! to the end if <exclaim> is set.
def sayHello [
  name: string,        # The person to say hello to
  --exclaim(-e): bool, # Whether to add a ! at the end
] {
  echo $"Hello, ($name)(if $exclaim { "!" })"
}

Now here’s the help output you get by running sayHello --help:

Say hello to <name> and add a ! to the end if <exclaim> is set.

Usage:
  > sayHello {flags} <name>

Flags:
  -e, --exclaim - Whether to add a ! at the end
  -h, --help - Display the help message for this command

Parameters:
  name <string>: The person to say hello to

As you can see, custom commands in Nushell feel like well-crafted CLI tools in their own right. This makes Nushell-based environments far more introspectable than their Bash counterparts.

How I created the environment

To create my new build environment for Nix, Nuenv, I first wrote a Nix function wrapping Nix’s built-in derivation function:

{
  mkNushellDerivation = {
    nushell,       # A Nushell package
    name,          # The name of the derivation
    src,           # The derivation's sources
    system,        # The host system
    packages ? [], # Same as in stdenv
    build ? "",    # Same as in stdenv
  }:

  derivation {
    inherit build name packages src system;
    builder = "${nushell}/bin/nu";
    args = [ ../nuenv/bootstrap.nu ];

    # Attributes passed to the environment
    # (prefaced with __nu_ to avoid naming collisions)
    __nu_builder = ../nuenv/builder.nu;
    __nu_nushell_version = nushell.version;
    __nu_envFile = ./env.nu; # Helper functions
  };
}

The basic mechanics: Nushell (set as the builder instead of Bash) runs a script called bootstrap.nu that performs some setup actions (like making the Nushell executable itself discoverable in the env). The bootstrapper then runs a builder.nu script that performs the actual realisation. As in all Nix derivations, the other attributes of the derivation are set as environment variables available to that script (plus some others that Nix provides). The builder.nu script needs to do quite a lot, including:

• Creating the directory in the Nix store where build output will end up (Nix provides an out environment variable for that but it doesn’t automatically create that directory).
• Copying all of the derivation’s sources (src or srcs in the standard environment) into the build sandbox, which resides in a temporary directory. Nix does put those sources in the Nix store at /nix/store/$HASH-source but the builder script needs to copy them into the sandbox.
• Making sure that the sandbox can discover any packages required by the derivation process using the PATH.

That establishes a kind of bare minimum environment. Beyond that, a builder script should provide things like helper functions (commands in Nushell) for use in derivation logic. The standard environment provides many such functions via the user-env.nu script, like wrapProgram and substituteInPlace. For Nuenv, I’ve provided two so far:

• substitute, which performs string substitution in a specified file and writes the output to a new file.
• substituteInPlace is like substitute but rewrites the specified file in place.

In principle, however, the full battery of stdenv functions could be replaced with custom Nushell commands. Overall, I was extremely impressed with how elegant the Nushell build script is compared to what its Bash equivalent would be. I’ll provide an illustrative example. Here’s how I set the PATH to discover packages in the Nix store:

# Parse the __nu_packages environment variable string, where each
# package path is separate by a space, and convert to a list
let packages = ($env.__nu_packages | split row (char space))

# Convert the list to a colon-separated string
let $packagesPath = (
	$packages
	| each { |pkg| $"($pkg)/bin" }
	| str collect (char esep)
)

# Set the PATH
let-env PATH = $packagesPath

Here’s how you would accomplish something like that in Bash:

export PATH=
for i in $packages; do
    if [ "$i" = / ]; then i=; fi
    PATH=$PATH${PATH:+:}$i/bin
done

As you can see, Nushell feels closer to something like Ruby.

Despite wins like this, Nuenv in its current state does have some major shortcomings vis-à-vis the Nixpkgs standard environment:

• There’s only one realisation phase, called build. By contrast, the stdenv is much more nuanced, offering support for Makefiles, configure scripts, and differentiated phases like buildPhase, installPhase, configurePhase, and checkPhase.
• You can only pass one set of packages to the environment, whereas stdenv makes a distinction between buildInputs, nativeBuildInputs, propagatedBuildInputs, and others. The dependencies you supply using packages in Nuenv are only available at build time and aren’t included in the runtime package, while stdenv enables you to target dependencies to both build time and run time as well as things like cross-compilation (which is not supported in Nuenv).
• Nuenv-based realisations are likely to be a bit more costly, in terms of build time and disk space used by the Nix store, than stdenv.mkDerivation because the Nushell package is required, although caching and the fact that Nushell doesn’t require coreutils should make this cost relatively small. A related limitation is that you can only realise Nuenv-based derivations on systems that can run Nushell. While that should be a fairly sizable range of systems given that Nushell is built in Rust, it’s nowhere near the range of systems that can run Bash.

So definitely don’t use Nuenv for any serious purpose just yet.

User-facing benefits of Nuenv

Although Nuenv is still at an early stage, I think that it already has some nice advantages over stdenv:

• Inside your derivation logic, you get access to the broad Nushell feature set I talked about above.
• It has much prettier log output thanks to Nushell’s easy-to-use ANSI support. While this is more of a quality-of-life thing, it does liven things up a bit.
• You can try out the Nuenv environment locally by running nix develop .#nuenv if you clone the repo (or nix develop github:DeterminateSystems/nuenv#nuenv if you don’t want to clone the repo locally). This gives you access to Nuenv’s helper commands. Run substituteInPlace --help to see an example. As far as I know, there isn’t really a way to introspect the stdenv in this way. You can access Bash with stdenv’s functions provided by running nix-shell -p but those functions don’t provide help output. You can also see a list of custom commands available to Nuenv derivations by running nix run github:DeterminateSystems/nuenv#nuenv-commands.

Try it out

You can try out Nuenv in several ways. To build an example package using the environment:

nix build --print-build-logs "github:DeterminateSystems/nuenv"

# See the result
cat result/share/hello.txt

This is a pretty simple derivation that pipes a string to GNU hello and writes that output to the share directory in the build output.

You can also build your own package with Nuenv! Here’s an example flake.nix:

{
  inputs = {
    nixpkgs.url = "nixpkgs";
    nuenv.url = "github:DeterminateSystems/nuenv";
  };

  outputs = { self, nixpkgs, nuenv }: let
    overlays = [ nuenv.overlays.default ];
    systems = [
      "x86_64-linux"
      "aarch64-linux"
      "x86_64-darwin"
      "aarch64-darwin"
    ];
    forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f {
      inherit system;
      pkgs = import nixpkgs { inherit overlays system; };
    });
  in {
    packages = forAllSystems ({ pkgs, system }: {
      default = pkgs.nuenv.mkDerivation {
        name = "hello";
        src = ./.;
        # This script is Nushell, not Bash!
        build = ''
          "Hello" | save hello.txt
          let out = $"($env.out)/share"
          mkdir $out
          cp hello.txt $out
        '';
      };
    });
  };
}

If you’ve installed Nix with flakes enabled, you can copy this flake.nix into a directory and run nix build && cat ./result/share/hello.txt to see the result. To run a Nushell script instead of providing a raw string, you can supply build = builtins.readFile ./my-nushell-script.nu to the derivation.

Ramifications

I think that using a more powerful builder for Nix—maybe Nushell, maybe something else—has major implications:

• If can offer a dramatically improved developer experience around derivation. Right now, derivation remains a bit of a dark art and one of the more intimidating aspects of using Nix (which is already intimidating in itself).
• It can radically alter the division of labor between Nix and the builder. Because Bash isn’t terribly powerful, Nix has to do a lot of the work of making things palatable to Bash, like constructing complex strings. But with a more powerful shell, Nix can defer a lot of work to the builder. Wouldn’t it be nice to just use Nix to supply simple attribute sets and let the builder handle the rest?
• Because any Nix environment is just Nix, you could introduce something like Nushell piecemeal into the Nix dependency tree. Imagine replacing some core utilities in Nixpkgs with more robust, ergonomic equivalents or dramatically simplifying language-specific derivation wrappers.

While I don’t see Nushell or its ilk taking over the Nix world by storm any time soon, I do hope that this post gets you thinking about what a qualitatively better builder might mean for Nix.

Conclusion: an oh-so-worthy experiment

My work on Nuenv hasn’t yielded a particularly robust Nix environment just yet but it’s been a fantastic learning experience about some of the lower-level nitty gritty of how Nix works. If you’re looking to level up your Nix knowledge, I strongly encourage you to take on a similar project (or hack on Nuenv!). You may just end up blazing a fruitful new trail for yourself and others in the Nix community—or with a renewed appreciation for the tried-and-true standard environment.

Finally, using Nushell in conjunction with Nix has frankly sparked joy in a way that trusty old Bash just doesn’t (for me, at least). While that isn’t a very “engineeringly” justification for building something, there’s much to be said for working on things that compel a dramatic shift of energy and attention. We don’t have any concrete plans to take Nuenv much further at this time but we’d love to see how the community reacts.