Rebuilding an existing package

Imagine that you're using a traditional Linux distribution (Debian/Ubuntu for me), all the packages that you use are binary packages. A common situation is that we want to build from the source package to the binary package: sometimes this is called a source package rebuild. In Guix, it's a little different as we start with a source package, and binary packages are referred to as Substitutes. While most people use Substitutes as it's less resource intense - we can actually build all packages ourselves, as you would in distributions like Gentoo Linux. Let's jump into how to to build a package from Guix's archive.

We'll start by building GNU Hello as it's a simple program. Click on Figure 1 above to watch a video of the steps in this section.

Our initial step is to create a build environment using guix shell:

➊ $ guix shell --container --nesting --development hello --network --no-grafts nss-certs

❷ # show the packages that are installed
  [env]$ echo $PATH
  /gnu/store/shk50v0w58f2f922p9rqra2kyg8ybsg4-profile/bin:/gnu/store/shk50v0w58f2f922p9rqra2kyg8ybsg4-profile/sbin

  [env]$ guix package --profile=/gnu/store//gnu/store/shk50v0w58f2f922p9rqra2kyg8ybsg4-profile --list-installed
  guix                    39ca9a964d7f45878295efc142be8abf5c7910eb        out     /gnu/store/c6vqmz7lvp18h7n6r3d67sgkni133m9q-profile
  linux-libre-headers     5.15.49                 out     /gnu/store/5iklcps70c0sfkxvlrhg8jhf3q4h18bj-linux-libre-headers-5.15.49
  glibc-utf8-locales      2.35                    out     /gnu/store/visfdda934gvivwihwhlm63fdqhhcc8a-glibc-utf8-locales-2.35
  glibc                   2.35                    static  /gnu/store/l0yryi5jsa1grnvw01c9nkz9c81cv224-glibc-2.35-static
  glibc                   2.35                    out     /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35
  gcc                     11.3.0                  out     /gnu/store/5lqhcv91ijy82p92ac6g5xw48l0lwwz4-gcc-11.3.0
  binutils                2.38                    out     /gnu/store/zh4x65snfis7svs6906gj1z8i7dx2j3m-binutils-2.38
  ld-wrapper              0                       out     /gnu/store/na1dpbbcxjaa3n8wkwrfpch476f90hlf-ld-wrapper-0
  bash-minimal            5.1.16                  out     /gnu/store/rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16
  make                    4.3                     out     /gnu/store/wj7casda7rb55rvqjnpm0bm7a2zm6618-make-4.3
  coreutils               9.1                     out     /gnu/store/a5i8avx826brw5grn3n4qv40g514505c-coreutils-9.1
  xz                      5.2.8                   out     /gnu/store/6k1yys9wqrfn4y41ic1win8gpnimncwj-xz-5.2.8
  grep                    3.8                     out     /gnu/store/yrv5f70mn83a876b78i5s79dd2hsh0zf-grep-3.8
  sed                     4.8                     out     /gnu/store/xxcfsimvxz7z4dj593gnqbkzc6picwzq-sed-4.8
  gawk                    5.2.1                   out     /gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1
  findutils               4.9.0                   out     /gnu/store/c8jyph2lxw0m9na34fg8h70n4nnnz7is-findutils-4.9.0
  patch                   2.7.6                   out     /gnu/store/210yfax18r2g2inxrml9435ikhfcca6m-patch-2.7.6
  diffutils               3.8                     out     /gnu/store/zmcf5kpqiighkbh7wslf91qdjwj06yr1-diffutils-3.8
  file                    5.44                    out     /gnu/store/gr0sy0m1mv36qv54idm6cn10l3mngshq-file-5.44
  bzip2                   1.0.8                   out     /gnu/store/j8wlfmlmfvpbza6is9wv9xsd8psrxn00-bzip2-1.0.8
  gzip                    1.12                    out     /gnu/store/x24bm49ag5dvki72mjdz195bfb89nrnb-gzip-1.12
  tar                     1.34                    out     /gnu/store/sxx22f98vfbavcqmdksm6as8fvskpxiw-tar-1.34
  nss-certs               3.88.1                  out     /gnu/store/5y39gqnvlfrw9gxyxbqqkdr8cxgp1fa1-nss-certs-3.88.1

We start by creating a shell container (at ➊) that has the ability to use the guix command (--nesting) as we'll need this for building packages. By using --development hello Guix installs all the dependencies that the hello package has, and any build tools (e.g. compilers) that are needed. The hello package doesn't have any dependencies itself, but it requires build tools. At ➋ we look at the PATH in the container, and then we list all the packages that have been installed - as we can see things like ld-wrapper and make are installed. Having all of these set-up and installed is such a time saver!

We also tell guix shell to install nss-certs so that we can use HTTPS later. If the options to guix shell are unfamiliar then check out the blog post series on guix shell.

The next step is to build the package:

  [env]$ guix build hello --no-substitutes
  The following derivation will be built:
      /gnu/store/xn12lv3ckihkai4044p28h5im78lvad5-hello-2.12.1.drv
  [... lots of output as it builds the package ...]
  successfully built /gnu/store/xn12lv3ckihkai4044p28h5im78lvad5-hello-2.12.1.drv
❸ /gnu/store/5mqwac3zshjjn1ig82s12rbi7whqm4n8-hello-2.12.1

We're using the guix build command to build from the source definition to the binary. We provide the package name ('hello') and tell it not to use a binary substitute (--no-substitutes) because we want it to build the package locally. The build command tells us what derivation will be built, then outputs the build log. Finally it (at ➌) it tells us it's successfully built the binary package and put it into the store (/gnu/store/5mqwac3zshjjn1ig82s12rbi7whqm4n8-hello-2.12.1).

This is the common two steps of package building - setting up a build environment with guix shell, and then building our package using guix build. Strictly speaking we don't need a build environment container but it avoids lots of problems to do this. It's particularly important when creating packages, as using a clean environment ensures there's no chance that we have a dependency in the runtime environment that's not included in our package definition.

⚠️ WARNING: If the build environment doesn't includes all the dependencies (using --development <package>) required then the build command will build them - this can be a huge waste of time and commonly we don't want it to do this.

Test the package by installing and using it (either in this environment, or into a separate test environment):

❹ [env]$ guix package --install /gnu/store/5mqwac3zshjjn1ig82s12rbi7whqm4n8-hello-2.12.1
  [env]$ /home/steve/.guix-profile/bin/hello --version

The locally built version of the hello package in our Store (/gnu/store/5mqwac...), which we can install by referencing it at ➍.

Common Guix Build Options

There are some common scenarios when building packages where guix build's options can help us out. Click on Figure 2 if you'd like to watch a video of the commands in this section.

If we try to build a package when we already have the binary in our Store it will fail:

  # exit from the current build environment
  [env]$ exit

  # start the build environment again and try to build
  $ guix shell --container --nesting --development hello --network --no-grafts nss-certs

❺ [env]$ guix build hello --no-substitutes
  /gnu/store/5mqwac3zshjjn1ig82s12rbi7whqm4n8-hello-2.12.1

What's happening at ➎ is that Guix knows we've already built the package so it just points us to the "one you did earlier". Normally, for users this is great as we don't want to download and build something if we already have it - but when we're rebuilding and playing with packages it's not what we want, to solve it we delete the binary package from the Store:

[env]$ guix gc --delete /gnu/store/5mqwac3zshjjn1ig82s12rbi7whqm4n8-hello-2.12.1
finding garbage collector roots...
[0 MiB] deleting '/gnu/store/q5gwi7p69zq67zsw2hwrzp08hcv9cvir-profile.drv'
[0 MiB] deleting '/gnu/store/1hnj4fqa1k68z8xf0zs7am

# now we can rebuild
[env]$ guix build hello --no-substitutes --no-grafts

One gotcha to look out is that if the package has been installed into any profile then you won't be allowed to delete for the obvious reason that some user is using it. This is another good reason to use a guix shell for test installing and playing with builds as when you leave the environment the profile is released. I've been caught a few times where I've installed a local package build into a profile and then had to go and find it in an old generation so that I could delete it.

At this point you probably want to rebuild a bunch of things, just because why not! Lets try a few more just for fun:

guix shell --container --nesting --development vifm --network nss-certs
guix build vifm --no-substitutes

guix shell --container --nesting --development fzf --network nss-certs
guix build fzf --no-substitutes

The resources needed to build an individual package is a function of it's complexity, dependencies and the language/tools that the build uses. In my experience:

• There are lot of smallish C programs that are easy to build (e.g. terminal games)
• Large applications like Firefox and OpenOffice can take a lot of resources and memory to build
• Modern languages like Python, Rust and Go tend to use many more libraries which can complicate builds

Every package in Guix should have a reproducible build - so everything should build - but before you decide to rebuild everything in the Guix archive it's worth considering whether it's a good use of your time, or the computers use of energy! And, as we'll learn the final results of a binary build will always be the same as the Substitute servers.

The guix build command has lots of options - the main ones that are useful at this point are:

Option	Description
--dry-run	Tell us what will happen but don't do it
--no-substitutes	Do not use binary substitutes
--no-grafts	Do not put security grafts onto the package and dependencies depending on context. Only needed for significant amounts of dependencies we want to speed up a build.
--verbosity=<level>	0=no output; 1 for quiet; 2 for show urls; 3 show build log
--cores=n	Use N CPU cores for each build job. Providing no value (0) means use as many cores as possible. This is the default.
--source	Build a source package
--max-jobs=N	Allow the Guix Daemon to run at most N build jobs. Each build job can be limited to use --cores=n.

The --source option is useful when we want to have the source of a package locally, so that we can build later e.g. when on a slow Internet connection. Here's how we get the source of cbonsai locally:

$ guix build cbonsai --source
downloading from https://ci.guix.gnu.org/nar/lzip/wzvywla078ksd1h68ydh1i526v042w9g-cbonsai-1.3.1-checkout ...
 cbonsai-1.3.1-checkout  21KiB
/gnu/store/wzvywla078ksd1h68ydh1i526v042w9g-cbonsai-1.3.1-checkout

When we perform a build it's actually done by the Guix daemon - we can see the Guix daemon accepting build tasks by using systemctl status guix-daemon.service. The --max-jobs option defines the number of builds that the Guix daemon will run in parallel (the special value 0) means no local builds. For example, if we were building a package that had multiple dependencies we could specify that we want to allow the daemon to build two packages simultaneously with --max-jobs=2. The --cores option defines how many cores each build job can use. If we don't provide a value then the Guix Daemon will try and use as much of the system as possible. Note that this is the number of cores per build, so if we ran --max-jobs=2 and --cores=3, each build will use 3 cores meaning a total of 6 cores will be used. Adding that to our cbonsai build we'd get something like this:

$ guix build cbonsai --max-jobs=2 --cores=4

Those are the most important options for rebuilding packages, we'll look at others when we learn how to customise packages.

Challenging builds

In this section we're looking at verifying binary builds: click on Figure 3 for a video of this sections content.

Building large parts of the package database is time-consuming and inefficient. It takes my laptop hours to build Firefox and I can't do much else while it's doing that. If a package isn't customised then it's going to be bit-for-bit the same as the Substitute servers build: it's not really worth the energy use for all users to build exactly the same thing! Consequently, it's common to use binary packages (Substitutes in Guix terminology).

One reason that people build their own packages is the concern that the binary packages they download from their Linux Distribution have been compromised. What if an evil cracker breaks into the Guix data center and distributes a backdoored OpenSSH package which exposes everyone who installs it?

With Guix we can resolve this concern by testing whether the binary Substitute packages have been tampered with. As Guix is functional a build will output the same package if it's given the same inputs. Guix stores the hash of the package inputs. It uses this hash in the package's name which it uses to put it into the Guix Store. Consequently, every build of cbonsai around the world will have the same hash (and filename) as the one we did above!

"The directory name contains a hash of all the inputs used to build that package; thus, changing an input yields a different directory name" (Managing Software the Guix Way)

This is also how we can test whether a package has been tampered with. If our own build lands up with the same hash as the Substitute servers then the build hasn't been tampered with.

The guix challenge command lets us check whether:

• Multiple Substitute servers agree on a build
• A local build and all Substitute servers agree

As we built cbonsai above, we can use it as an example:

$ guix challenge cbonsai --verbose
/gnu/store/mgc2i6yxm2zbqf8yx8x5f4ig4nbii2cv-cbonsai-1.3.1 contents match:
  local hash: 1vws4ywn1gcgpnm1pfr5rz4hv769ccvnyj5drpnnway7bg0ckh28
  https://ci.guix.gnu.org/nar/lzip/mgc2i6yxm2zbqf8yx8x5f4ig4nbii2cv-cbonsai-1.3.1: 1vws4ywn1gcgpnm1pfr5rz4hv769ccvnyj5drpnnway7bg0ckh28
  https://bordeaux.guix.gnu.org/nar/lzip/mgc2i6yxm2zbqf8yx8x5f4ig4nbii2cv-cbonsai-1.3.1: 1vws4ywn1gcgpnm1pfr5rz4hv769ccvnyj5drpnnway7bg0ckh28

1 store items were analyzed:
  - 1 (100.0%) were identical
  - 0 (0.0%) differed
  - 0 (0.0%) were inconclusive

As we can see guix challenge identifies our local build and shows the local hash that the build has created. It then checks the ones on both Substitute servers that I'm using. In this case everyone agrees - so we can be confident that binary build hasn't been tampered with.

There's higher risk software - like anything that talks to the Internet, and particularly any server daemons. The problem with checking the builds of these is that they're often critical to our system - we can't remove them first, and then do a local build.

$ guix challenge openssh --verbose
/gnu/store/kz7ijzmv3qyr5nkgy296w87g8z4avmrh-openssh-9.4p1 contents match:
  no local build for '/gnu/store/kz7ijzmv3qyr5nkgy296w87g8z4avmrh-openssh-9.4p1'
  https://ci.guix.gnu.org/nar/lzip/kz7ijzmv3qyr5nkgy296w87g8z4avmrh-openssh-9.4p1: 1hzfn3a2jn4l2n8lvpp8lzs9yiag97nyggw1ps9xy44aqqypcps6
  https://bordeaux.guix.gnu.org/nar/lzip/kz7ijzmv3qyr5nkgy296w87g8z4avmrh-openssh-9.4p1: 1hzfn3a2jn4l2n8lvpp8lzs9yiag97nyggw1ps9xy44aqqypcps6

1 store items were analyzed:
  - 1 (100.0%) were identical
  - 0 (0.0%) differed
  - 0 (0.0%) were inconclusive

In this case, Guix identifies that I have not done a local build with the message "no local build for XXX", so it knows that while I have OpenSSH installed I didn't build it myself. However, it can check that the two Guix Substitute servers agree - which they do.

Maybe I'm still a bit concerned that someone has broken into the Guix servers. We can actually rebuild a package that we already have installed by using the --check option with guix build

$ guix shell --container --nesting --development openssh --network nss-certs
$ guix build openssh --no-substitutes --no-grafts --check
[... lots of build output ...]
successfully built /gnu/store/k7pyp2gk4hakmk10aa37xds62a1gjxxl-openssh-9.4p1.drv
successfully built /gnu/store/k7pyp2gk4hakmk10aa37xds62a1gjxxl-openssh-9.4p1.drv
/gnu/store/kz7ijzmv3qyr5nkgy296w87g8z4avmrh-openssh-9.4p1

The --check switch builds the package multiple times as one of it's goals is to test reproducibility. The default is to do two builds, we can specify this further with --rounds N.

Now when we do guix challenge:

guix challenge openssh --verbose

/gnu/store/kz7ijzmv3qyr5nkgy296w87g8z4avmrh-openssh-9.4p1 contents match:
  local hash: 1hzfn3a2jn4l2n8lvpp8lzs9yiag97nyggw1ps9xy44aqqypcps6
  https://ci.guix.gnu.org/nar/lzip/kz7ijzmv3qyr5nkgy296w87g8z4avmrh-openssh-9.4p1: 1hzfn3a2jn4l2n8lvpp8lzs9yiag97nyggw1ps9xy44aqqypcps6
  https://bordeaux.guix.gnu.org/nar/lzip/kz7ijzmv3qyr5nkgy296w87g8z4avmrh-openssh-9.4p1: 1hzfn3a2jn4l2n8lvpp8lzs9yiag97nyggw1ps9xy44aqqypcps6

1 store items were analyzed:
  - 1 (100.0%) were identical
  - 0 (0.0%) differed
  - 0 (0.0%) were inconclusive

This time Guix knows that we've performed a local build and shows the local hash that's been generated. We can see that all Substitutes servers and our own build agree - the binary Substitute has not been compromised!

⚠️ WARNING: Make sure guix challenge is done outside a container, otherwise it always says there's no local build.

As always with security it's important to know what is not being promised through this process. There are many vectors for vulnerabilities throughout the software supply chain. But, at least with Guix we can assure ourselves that binary packages can be verified with a strong promise.

Final thoughts

Hopefully building packages from Guix's source and package definition now feels straightforward. We can use this capability to check the verify the binary packages (guix challenge).

Rebuilding an existing package is our first scenario, from what we've learnt here we can move onwards to modifying packages.

This is the first post I've tried some videos in - it takes a while to create them - so please tell me if they're useful!