Accessing Network from a Nix Derivation (via Fixed output derivations)

Published by Philipp Schuster on 2024-03-192024-03-19

Usually, Nix derivations are build in a sandbox that prevent any program from accessing the network. This has a good reason: We want reproducibility in Nix, and using the network is not reproducible for many reasons! But there must be a way of accessing the network from a derivation, right? Why? Because, builtins.fetchGit or builtins.fetchTarball are not flexible enough to cope with the nature of Maven, Cargo, npm or other tools that download something from the network during their setup. Oh, and I’m not talking about disabling the Nix sandbox (which is possible).

The solution is to use fixed output derivations. IMHO, they are poorly documented, at least if you don’t already know what you are looking for. They are mentioned in the Nix glossary but appear only as a hidden side note in the nixpkgs manual (at least in March 2024).

Technically, a fixed output derivation is a normal derivation, but you also define a specific known output hash beforehand. The best practice to package such projects is to split your source processing, i.e., the aggregating of the sources, from the actual build into a dedicated derivation. This derivation then serves as src of the actual derivation, performing the actual build step.

Take a look at the following example. In this (very simplified) example, we want to package a project which contains a ./example_that_downloads_additional_sources.sh that needs to be executed before the actual build process. It downloads some additional resource that is needed to build the project:

let
  pkgs = import <nixpkgs> { };
  # Derivation that contains the original source and the aggregated source.
  aggregatedSource = pkgs.stdenv.mkDerivation {
    pname = "foobar-aggregated-source";
    version = "0.0.0";
    src = ./.;
    doCheck = false;
    dontFixup = true;
    nativeBuildInputs = with pkgs; [ cacert wget ];
    buildPhase = ''
      runHook preBuild
      ./example_that_downloads_additional_sources.sh
      runHook postBuild
    '';
    installPhase = ''
      runHook preInstall
      mkdir $out
      # File was downloaded from script in build phase. Copy it with the other
      # sources to $out. This depends on the nature of your project. Sometimes
      # you might only want to keep the downloaded artifacts.
      #
      # $out must not include anything that has a path into the Nix store,
      # otherwise you get 'illegal path references in fixed-output derivation'
      # errors!
      cp -r . $out
      runHook postInstall
    '';
    outputHashAlgo = "sha256";
    outputHashMode = "recursive";
    outputHash = "sha256-4SePc3yGlBTGCoCeZtVL9A1NK5vv2CM8EnoRCinhPA0=";
    # outputHash = pkgs.lib.fakeHash;
  };

#################
# Part two below
in

pkgs.stdenv.mkDerivation {
  pname = "foobar";
  version = "0.0.0";
  # Now we use the aggregated/processed sources.
  src = aggregatedSource;
  buildPhase = ''
    runHook preBuild
    # Do something useful ...
    runHook postBuild
  '';
  installPhase = ''
    runHook preBuild
    mkdir $out
    # Show that we processed the previously downloaded file. Usually, you would
    # do something more useful here!
    cp index.html $out
    runHook postBuild
  '';
}

That’s it. No dark magic involved here.