Guix shell overview

Everything for creating environments and containers in Guix is accessed through the guix shell command. There's a similar command called guix environment which is the predecessor, and some of the older documentation refers to it. The guix shell command is slightly easier to use and is the recommended way.

To use it we do:

guix shell [options] [packages...]

# example 1: start an environment adding the 'hello' command
$ guix shell hello
[env]$ hello
Hello, world!
[env]$ exit

To return to our previous 'normal' environment type exit or do CTRL-D: this might seem obvious but I always remember being stuck in Emacs for the first time and trying all sorts of key combinations to exit with increasing angst!

We can tell guix shell to install multiple applications by adding them to the command line. Normally it will drop us into a shell in the new environment, but we can also tell it to run a command directly:

# example 2: start a throw-away environment with multiple packages installed
$ guix shell bat dutree exa fd zoxide

# example 3: start an environment and run the htop tool directly
$ guix shell htop -- htop

In each example we're creating an ephemeral environment and adding the new packages to our existing $PATH: we can use all our usual commands (e.g. ls, vim, etc) along with the new ones we've added (e.g. hello in the first example).

We can see we're augmenting our existing environment by looking at the $PATH inside a guix shell .

[env]$ echo $PATH
/home/steve/.guix-profile/bin:/gnu/store/xgi7iiyh0v2vrwz206pmh6g7b10rrsgf-profile/bin:[... rest not shown ...]

$ ls /gnu/store/xgi7iiyh0v2vrwz206pmh6g7b10rrsgf-profile/bin/
hello*

As we can see guix shell creates a new environment and puts the new package(s) into a temporary profile which is added to our $PATH.

Bringing along our current environment is convenient as we can use all our tools and settings. However, it means the new shell is not a fully separated environment which can cause confusion. A good example of the potential for confusion is if we install Python into a guix shell environment it will have access to our machines site packages - the result is we might be using a library from the host, or one added to the guix shell environment.

The way to avoid this is to start a completely pristine environment using the --pure switch:

$ guix shell --pure hello
$ echo $PATH
/home/steve/.guix-profile/bin:/gnu/store/xgi7iiyh0v2vrwz206pmh6g7b10rrsgf-profile/bin

With the --pure switch the whole of our environment is cleaned and the only commands we have access to are those that we added through packages.

🏆 TIP: As a pure shell environment has no commands in it other than the packages that were specified I'll generally add the coreutils package so that I have some basic utilities.

Guix shell and manifests

Using manifests with guix shell makes it easy to create a clean virtual environment and install all the tools that we need. If we provide a manifest to guix shell (using --manifest=<file> it will install the listed packages as part of creating the environment.

📝NOTE: for an introduction to manifests see the managing guix profiles post.

As an extra trick if a file named manifest.scm is found in the current working directory, or any of the ancestors Guix will automatically use it without it being specified as part of the command.

For example, create a temporary directory ("project1") and a file called manifest.scm, put the packages from our second example in it:

(specifications->manifest
    '("bat"
      "dutree"
      "exa"
      "fd"
      "zoxide"))

As a security measure Guix will only use the manifest.scm automatically if the directory that it's in has been authorised. When we run guix shell it detects a manifest.scm and tells us how to authorise this directory:

echo /home/steve/workspace/project1 >> /home/steve/.config/guix/shell-authorized-directories

Now when we create our shell environment we see the manifest is being used:

$ guix shell
guix shell: loading environment from '/home/steve/workspace/project1/manifest.scm'

There's a related capability using a guix.scm file, we'll cover that in our next post.

Guix shell and containers

To guarantee isolation guix shell can create a container. The overhead on a container is very low and it guarantees separation, so it's my preferred way of using guix shell. It's also useful for running software that we don't completely trust. To create a container we add the --container option:

$ guix shell --container coreutils

Just like when using --pure a container environment is very clean. Try running the env command in your normal environment and then inside a guix shell container to see the significant difference!

Preserving environment variables

Completely removing all the environment variables that come from the parent environment often isn't useful: for example, almost nothing will run without the $TERM environment variable. We can use the --preserve switch to keep the environment variables that we want. Here are a couple of simple examples:

# example 4: start htop to see resources used in the container
$ guix shell --container --preserve='^TERM$' coreutils htop -- htop

# example 5: start the vifm file manager
$ guix shell --container --preserve='TERM$' coreutils vifm vim git -- vifm

In example 4 the htop shows only the processes in the container as it's a constrained environment.

As the sophistication of the application increases we often have to preserve multiple different parts of the environment - we can provide the --preserve option multiple times to accomplish this.

🏆 TIP: It can often take some messing around to find the minimum environment variables that an application needs, to get things running initially we can preserve everything by providing --preserve=$.

Accessing the network

The default is that a container will not have access to the network, provide the --network switch so that the network is available.

For our first example lets explore the non-web protocols like Gemini and Gopher with Bombadillo :

# example 6: using the gemini protocol
$ guix shell --container --preserve='^TERM$' --network bombadillo -- bombadillo

# example 7: using some network utilities
$ guix shell --container --preserve='^TERM$' --network coreutils traceroute -- traceroute guix.gnu.org

As pretty much everything uses SSL these days there's often a need to mess with this to make it work. Some applications will find the required certs if we share the /etc/ssl/certs directory from the host, but not all. The Guix Manual has a section on X.509 Certificates that explains the details. For some we might have to figure out the right environment exports, here's an example using a Python based tldr command:

# example 8: providing SSL certs
$ guix shell --container --network --expose=/etc/ssl/certs/ python-tldr coreutils
[env]$ export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
[env]$ tldr ls

XWindows Sharing

So far our examples have been command line tools, to use a graphical application at a minimum we need to share the $DISPLAY and provide permission for the application to connect to it (XAUTHORITY).

This is a simple example of using the gitk visual viewer, to view a clone of the Guix git repository

# example 9: using gitk
$ guix shell --container --network --preserve='^DISPLAY$' \
--preserve='^XAUTHORITY$' --expose=$XAUTHORITY --expose=/etc/ssl/certs git git:gui nss-certs coreutils
[env]$ git clone https://git.savannah.gnu.org/git/guix.git guix-tmp
[env]$ cd guix-tmp
[env]$ gitk

For more complex applications that are using Desktop environments (e.g. GNOME or KDE) it may be necessary to share things like DBUS - there's an example of that in a moment.

File system Sharing

When we start a container it will share the current working directory, we can prevent this with the --no-cwd switch. Similarly, we can avoid sharing our GECOS data by using a fake user through the --user=<user> option.

# example 10: hiding GECOS and current workng directory
$ guix shell --container --no-cwd --user=Bob coreutils

[env]$ cat /etc/passwd
Bob:x:1000:1000::/home/Bob:/gnu/store/m6c5hgqg569mbcjjbp8l8m7q82ascpdl-bash-5.1.16/bin/sh

[env]$ pwd
/home/Bob

I honestly haven't come up with a circumstance where I wanted to hide my GECOS data, but running in a clean share is pretty useful. Which brings us onto how to expose and share parts of the disk to the container.

We can use the --expose option to share part of the filesystem as read only, if we want to allow read/write access we use the --share switch. If we just provide the path then that's how it will be made available in the container. Alternatively, we can specify how we'd like the location mounted (/some/dir=/mount/inside/here). For example:

# example 11: mounting a system directory (/usr/local)
$ guix shell --container --expose=/usr/local util-linux grep coreutils
[env]$ mount | grep /usr/local
/dev/nvme0n1p3 on /usr/local type btrfs (ro,relatime,compress=zstd:1,ssd,discard=async,space_cache=v2,autodefrag,subvolid=256,subvol=/rootfs)

# example 12: mounting a directory as home
# create a temporary directory and mount this as a directory called guix-home
$ mkdir -p ~/tmp/guix-fakehome
[env]$ guix shell --container --no-cwd --share=$HOME/tmp/guix-fakehome=/home/steve/ coreutils
[env]$ pwd

🏆 TIP: it's often useful to prevent the current working directory from being shared (--no-cwd), create a temporary home for the environment (--share=/tmp/<whatever>=/home/<user>) and then only share the specific directories the application needs such as runtime configuration from ~/.config.

Firefox in a container

We now have all the options that we need in order to run a complex graphical application in a Guix container. I've created a guix-firefox script and it has:

#!/usr/bin/env bash

guix shell --container --network --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --expose=$XAUTHORITY \
--preserve='^DBUS_' --expose=/var/run/dbus --expose=/etc/ssl/certs/ --expose=/dev/dri \
--share=/dev/snd/seq --share=/dev/shm --expose=/sys/class/input --expose=/sys/devices \
--expose=/sys/dev --expose=/sys/bus/pci \
--expose=/run/user/"$(id -u)"/pulse --preserve='XDG_RUNTIME_DIR' --share=$HOME/.config/pulse \
--share=$HOME/.mozilla --share=$HOME/Downloads --no-cwd firefox intel-vaapi-driver dbus-glib \
pciutils alsa-lib pulseaudio -- firefox &>/dev/null &

It's quite a lot of effort to get to something that works, there was a lot of trial and error to reduce how much was being shared. As Firefox is a Guix application it wasn't too difficult to find out what is needed to run it - as I have an Intel card I'm specifically providing the driver. To debug things as I was going along it was really useful to have core-utils and mesa-utils (for glxinfo -B).

Figure 2 (click to enlarge): Bottom a resource viewer in a Guix Shell

If an application isn't in the Guix archive and can't be compiled from source we need guix shell to create a standard FHS compatible environment.

FHS

The Filesystem Hierarchy System (FHS) is a standard that all Linux distributions follow so that binaries know where to find common requirements such as where information about devices is, or where system libraries are located.

However, Guix doesn't obey the FHS: this is due to the way that Guix links everything in a profile to a specific (hashed) build of a package. This isn't an issue for software that is compiled as a Guix package, because all the information is provided as part of the package definition. But, software that ships as binaries such as commercial games, or enterprise software won't work.

Luckily Guix shell has a trick up its sleeve where it can create an environment that obeys the FHS. It does this by creating links so that all the standard directories are where you'd expect. We can use this capability to run compiled binaries.

A simple example is bottom a process viewer. In this example we'll download the compiled binary and run it in a container using the --emulate-fhs flag.

$ wget https://github.com/ClementTsang/bottom/releases/download/0.8.0/bottom_x86_64-unknown-linux-gnu.tar.gz
$ guix shell --container --network --emulate-fhs libgccjit coreutils - btm
[env]$ ./btm

See Figure 2 for what it looks like running in the container.

For many people games are an interesting way to use containers. For an example I picked Age of Conquest IV a turn-based strategy game by Indie game developers Noble Master Games - this is a commercial game, the first map is free. This is a good game to get running because it creates an install log where it tells you what's missing.

To run it we're using a script that is both a shell command and a Guix manifest: originally suggested on the Guix devel mailing list by Kaelyn. Having both a script and a manifest at the same time is a super clever hack!

The script is:

#!/usr/bin/env bash
set -ex
exec guix shell --verbosity=3 --container --network --emulate-fhs --preserve='^DISPLAY$' \
--preserve='^XAUTHORITY$' --expose=$XAUTHORITY --preserve='^DBUS_' --expose=/var/run/dbus \
--expose=/dev/dri --share=/dev/snd/seq --share=/dev/shm --expose=/sys/class/input \
--expose=/sys/devices --expose=/sys/dev --expose=/sys/bus/pci --expose=/run/user/"$(id -u)"/pulse \
--preserve='XDG_RUNTIME_DIR' --manifest="$0"
!#

(use-modules (gnu))
(manifest
 (map (lambda (spec)
        (apply package->manifest-entry
               (cond
                ((pair? spec) (let ((pkg (car spec))
                                    (output (cadr spec)))
                                (list
                                 (if (string? pkg)
                                     (specification->package pkg)
                                     pkg)
                                 output)))
                ((string? spec) (list (specification->package spec)))
                (else (list spec)))))
      `("bash" ; always required
        "coreutils" ; always required
        "glibc" ; always required
        (,(@@ (gnu packages gcc) gcc) "lib") ; workaround to get gcc-lib
        "firefox"
        "intel-vaapi-driver" ;specific to my set-up
        "openjdk"
        "mesa"
        "sed"
        "gawk"
        "libxrender"
        "libxtst"
        "zlib" ;log says libz is needed
        "bzip2"
        "lbzip2" ; log says libbz2 is needed
        "alsa-lib"
        "pulseaudio"
        "nss-certs"
        "util-linux"
        )))

To explain how this works, the first section a normal guix shell command. Notice that it refers to the manifest $0 - this refers to itself. When the guix shell command runs, it feeds in the script and guix sees it as a manifest (in the language Guile Scheme) so ignores everything up to the second !# - as it sees this as a block comment.

The second part of the script is a manifest written in Guile. The reason this section is needed is to install gcc's lib, which is currently a hidden package in Guix. By the time you read this a later version of Guix may have made this lib easy to access. The rest of the packages are the ones needed to access and use 3D along with some utilities the installer needs.

To run it we do:

$ ./guix-conquest.sh
[env]$ export LD_LIBRARY_PATH=/lib:/usr/lib
[env]$ cd com.ageofconquest.app.user.aoc/
[env]$ ./conquest

Figure 3 (click to enlarge): Age of Conquest in a Guix Shell!

The FHS capability is very promising as it gives us a way to repeatedly creating an environment to run proprietary applications. That said it's quite raw - finding two programs that would work for the examples in this section took me hours and hours of effort. I tried a few different applications and had to give up with some of them.

The problem with compiled binaries is that they are opaque. If the application won't run in the container then you have to fall back on tools like ldd, strace and gdb which is pretty desperate. Age of Conquest logs what's missing, but it was the only one that I found that did this - many of the others just failed with an inscrutable error message.

Guix shell resources

If you'd like to explore guix shell more here are some additional resources.

• From 'guix environment' to 'guix shell'

  Covers why the guix shell command was created.

• Guix manual - invoking guix shell

  Comprehensive coverage - there are a few parts I found a bit dense, so read all the words carefully!

• The Filesystem Hierarchy Standard Comes to Guix Container

  Blog post covering the FHS feature of guix shell.

• Using GNU Guix Containers with FHS (Filesystem Hierarchy Standard) Support

  In this video John Kehayias, the developer of the FHS capability, covers how to to use it.

Summary

The ability to create clean environments declaratively is very useful for limiting interactions between different libraries and reducing how much of the system is shared. Starting a container is so easy and there's so little overhead - my fingers almost do it automatically without me thinking now!

It's also great for ensuring reproducibility, as we can create exactly the same environment every time. For developing software this is an important aspect. In the next post we'll dive into using Guix shell to create reproducible environments with all the tools and capabilities that developers need on a per project basis.