Layered Profiles

Technically, it's possible to activate multiple profiles at the same time. However, I've found this doesn't work very well, so I don't advise it. The problem is how environment variables interact when multiple profiles are activated.

When activating a profile various environment variables are altered depending on the applications being installed. The obvious one is that the $PATH is altered to add the directory where the profile has installed its binaries. Lots of applications need their environment variables to determine the locations of runtime components like plugins and modules. To activate multiple profiles simultaneously we need to merge the locations these environment variables point to: for example, if we activate two profiles the $PATH needs to include the bin directory of both profiles.

Some applications don't understand the concept of an environment variable having multiple directories in it, or one profile's activation wipes out the other profiles environment variable. I've had lots of failures from trying this. For example, I had problems with Jupyter because if you install Python packages in two or more profiles then you need to merge PYTHONPATH so that the modules in different locations can be found.

The solution is to merge the different manifests together into one profile, that way we're installing all the applications into that single profile and all the environment variables are part of it. This is possible because a Guix manifest is a GNU Guile file, this means we can use Guiles capabilities to create the profile. Mentally, I think of this as a set of layers that are combined together into a singe layered profile.

For my situation I have a set of manifests that I use when I'm working in a shell environment (base-shell.scm, util-shell.scm and dev-shell.scm) and a similar set for working in a graphical environment (base-Xwin, util-Xwin and dev-Xwin). When I'm logging into an environment initially I load base-shell which has the smallest set of essential utilities. When I SSH into my main workstation I want all the shell utilities to be available so I've created a all-shell.scm which brings all three together into a single layered profile:

;; A combined shell profile combining everything in base, util and dev
;; create-profile all-shell
;; activate-profile all-shell
;; See: https://lists.gnu.org/archive/html/help-guix/2018-10/msg00040.html
(use-modules (srfi srfi-1)
             ((guix ui) #:select (make-user-module)))

(define (load-manifest file)
    ;; Load manifest file in a fresh module with necessary imports.
    (let ((module (make-user-module '((guix profiles) (gnu)))))
        (save-module-excursion
            (lambda _
                (set-current-module module)
                (load (canonicalize-path file))))))

(define (combined-manifest-from-files . files)
    (fold (lambda (file combined)
            (manifest-add combined
                          (manifest-entries (load-manifest file))))
          (manifest '())
          files))

;; list the full path of the manifests we're loading - not a shell so
;; doesn't accept things like $HOME
(combined-manifest-from-files
   "/home/steve/.config/guix/manifests/base-shell.scm"
   "/home/steve/.config/guix/manifests/util-shell.scm"
   "/home/steve/.config/guix/manifests/dev-shell.scm"
     )

; vim ft=scheme

The main part is the bottom section, containing the call to combined-manifest-from-files, here we specify manifests that we want to combine together. As these are being combined by Guile we have to provide the full path to the manifest file (i.e. shell variables like $HOME are not available). The credit for this code goes to Mikhail Kryshen who came up with this solution on the Guix Help mailing list thread using multiple manifest files.

With this concept we can split our package manifests and combine them together how ever we'd like. For example, create a core set of packages, and then layer on particular server capabilities or project capabilities. Use what ever mental organisational model works for you!

Profiles at Login

The last element is activating the needed profiles at login so that all the applications are available. Again, rather than activating multiple profiles, I'm sourcing a single layered profile.

We have a layered profile that consists of all the shell applications so in ~.bash_profile I have the following:

GUIX_PROFILE="$HOME/.guix-extra-profiles/all-shell/all-shell"
if [ -d  $GUIX_PROFILE ]; then
    # rather than downloading everything we just source it
    . "$GUIX_PROFILE/etc/profile"
fi

I tried using the create-profile and activate-profile scripts that I showed in the previous blog post, but this slows logging in down as new package versions are downloaded and installed. Consequently, here we're simply sourcing in the version of the profile that has already been created.

Most of the time I login through the graphical greeter, I have the same script element in my ~/.xprofile.

Summary

With layered profiles and the scripts from the previous posts we have all the tools that are needed to manage and deploy the same applications into all our environments and projects. The key mental realisation for me is manifests and profiles are perfectly paired: a manifest is instantiated as a profile, and a profile is simply a runtime environment created from a manifest.

We can have multiple manifests to create multiple profiles, each specific to a use-case. With the concept of layered profiles we can change the model so that we have multiple manifests combined together into a profile - making it much more flexible and efficient to maintain manifests.