Nix Overlays: Add attribute to “lib” and avoid “infinite recursion error”

Update 2024-01-01: Renamed self and super to final and prev as recommended by the Nix conventions. Using this names makes using overlays much easier to understand.

Original Post: Today, I was about to add an attribute to the lib attribute of nixpkgs using a nixpkgs overlay. I thought that this is as straight forward as adding regular packages directly in an overlay. This wasn’t my first time using overlays, but my first time adding something to lib. There was a tiny but ugly pitfalls, that made me create this short blog post.

Let’s look at the following code foo.nix:

# This script can be invoked like this: $ nix-build ./foo.nix
let
  # Overlay that adds functionality to "pkgs.lib".
  overlay = (_final: prev:
    let
      pkgs = prev.pkgs;
      lib = prev.lib;
      toPretty = lib.generators.toPretty;
      # Function that takes a value to trace and an expression to return. The
      # semantics is the same as for builtins.trace, but the traced value is
      # prettified before being printed.
      tracePretty = val: ret: (builtins.trace (toPretty { } val) ret);
      # Wraper around tracePretty that traces the given value and returns it.
      tracePrettyVal = val: tracePretty val val;
    in
    {
      # Must be prev.lib and not prev.pkgs.lib, otherwise: infinite recursion
      lib = lib // {
      	inherit tracePretty tracePrettyVal;
      };
    }
  );
  # pkgs with the applied overlay.
  pkgs = import <nixpkgs> { overlays = [ overlay ]; };
in
# Usage of the overlayed attribute.
pkgs.lib.tracePrettyVal { arr = [ 1 2 3 ]; }

With the top part (overlay = ...), I define a basic overlay that adds the functions tracePretty and tracePrettyVal to a lib attribute that is merged with prev.lib. We must be very careful that we merge it with prev.lib but not prev.pkgs.lib. As I occasionally write Nix files like this:

{ pkgs ? import <nixpkgs> {}
, lib ? pkgs.lib
}: ...

I was tricked to think that I have to use prev.pkgs.lib. However, actually, prev refers to what pkgs is in the snippet just mentioned and prev.pkgs.lib refers to pkgs.pkgs.lib. So, be careful that you merge your library functions with prev.lib!

The rest of the code in the beginning of the blog post is just necessary clutter to use a Nix overlay. So, as a summary: Write an overlay that adds something to lib like this:

(final: prev: {
  lib = prev.lib // {
    foo = "bar";
  };
})

Otherwise, when you accidentally use prev.pkgs.lib, you end up with an ugly “infinite recursion error”.

TIL

Today I learned that pkgs = import <nixpkgs> {} also has a pkgs.pkgs sub-attribute, which is a reference to the top-level pkgs. So you can use pkgs.pkgs.pkgs..... I use Nix for a few months now, but I never encountered this. However, the problem above shows that overlays should always add stuff to the top layer.