OpenTelemetry tracing for Determinate Nix

We’re excited to announce that Determinate Nix can now export OpenTelemetry (OTel) traces of just about everything it does, from evaluation and local builds to cache substitutions, HTTP requests, remote builds, and Nix daemon connections. It sends them over the standard OpenTelemetry Protocol (OTLP), so you can inspect them in any backend that accepts OTLP over HTTP, including Honeycomb, Grafana Tempo, Jaeger, and Datadog.

Here’s an example visualization in Honeycomb of a nix build run that includes calls to the Nix daemon as well as a store path substitution:

Expand image

Tracing provides a wealth of information that you can use to answer questions like “why is this CI run involving Nix slower than expected?” At Determinate Systems, we’re already using it to diagnose performance issues, and in the future we envision it being a constant companion in our efforts to make Determinate Nix the fastest and most reliable distribution of Nix. But this feature isn’t just about us: we’re eager to see how Determinate Nix users put tracing to work in their own workflows.

Why OpenTelemetry

We chose OpenTelemetry because it provides a vendor-neutral standard for collecting logs, metrics, and, in our case, traces, which in turn gives Determinate Nix users—including us!—with the widest available range of vendors and tools to choose from.

We also like OpenTelemetry because it provides truly distributed tracing rather than just a local profiler for the Nix CLI. In accordance with the OpenTelemetry specification, Determinate Nix propagates the W3C trace context to the Nix daemon, remote builders, and binary caches, which means that a complex, multi-phase operation like nix build is represented as a single trace, no matter how many processes it touches (you can see an example of this in the Honeycomb example image above). So if your workflow already emits traces, you can set the TRACEPARENT environment variable to make Determinate Nix one service amongst others.

It’s important to note that tracing is turned off by default in Determinate Nix, so no tracing information leaves your machine until you enable it and point Nix at a collector. Check out our OpenTelemetry for Determinate Nix documentation for a wealth of information about configuration, authentication, and sampling, as well as an exhaustive account of what Determinate Nix puts in its spans.

Give it a try

A nice way to try this out locally is to use otel-desktop-viewer, a single binary that accepts OTLP and shows your traces in the browser:

nix run "https://flakehub.com/f/NixOS/nixpkgs/0.2605.1014769#otel-desktop-viewer"

In another terminal, run any Nix command with an OTLP endpoint set:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 \
  nix build --no-link "https://flakehub.com/f/NixOS/nixpkgs/0#cowsay"

Then open http://localhost:8000 to explore the nix build trace.

If you don’t have the most recent Determinate Nix installed, you can try it like this:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 \
  nix run "https://flakehub.com/f/DeterminateSystems/nix-src/3.23.0" -- \
    build --no-link "https://flakehub.com/f/NixOS/nixpkgs/0#cowsay"

How to get Determinate Nix

If you already have Determinate Nix installed, you can upgrade to 3.23.1 with one Determinate Nixd command:

sudo determinate-nixd upgrade

If you don’t yet have Determinate Nix installed, you can upgrade or migrate to Determinate Nix on macOS using our graphical installer:

Install Determinate Nix on macOS

With support for Apple Silicon (aarch64-darwin)

On Linux:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
  sh -s -- install --determinate

On NixOS, we recommend using our dedicated NixOS module or our NixOS ISO (NixOS installer for x86_64, NixOS installer for ARM) with Determinate Nix pre-installed.

On GitHub Actions:

on:
  pull_request:
  workflow_dispatch:
  push:
    branches:
      - main

jobs:
  nix-ci:
    runs-on: ubuntu-latest
    # Include this block to log in to FlakeHub and access private flakes
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@main
      - uses: DeterminateSystems/flake-checker-action@main
      - uses: DeterminateSystems/determinate-nix-action@v3
      - uses: DeterminateSystems/flakehub-cache-action@v3
      - run: nix flake check

In Amazon Web Services:

data "aws_ami" "detsys_nixos" {
  most_recent = true
  owners      = ["535002876703"] # Commercial
  # owners      = ["579351485434"] # GovCloud
  # owners      = ["129194717446"] # European Sovereign Cloud

  filter {
    name   = "name"
    values = ["determinate/nixos/epoch-1/*"]
  }

  filter {
    name   = "architecture"
    values = ["x86_64"]
  }
}