plicitly declared in the derivation being built. NixOS makes one exception: it relies on the availability of /bin/sh in the chroot.[1] We remove that exception, and instead automatically patch script "shebangs" in the package’s source, as noted in Section 3.4. This turned out to be more than just a theoretical quest for "purity". First, some GNU/Linux distributions use Dash as the implementation of /bin/sh, while others use Bash; these are two variants of the Bourne shell, with different extensions, and in general different behavior. Second, /bin/sh is typically a dynamically-linked executable. So adding /bin to the chroot is not enough; one typically needs to also add /lib* and /lib/*-linux-gnu to the chroot. At that point, there are many impurities, and a great potential for non-reproducibility—which defeats the purpose of the chroot.
Several packages had to be adjusted for proper function in the absence of /bin/sh.[2] In particular, libc’s system and popen functions had to be changed to refer to "our" Bash instance. Likewise, GNU Make, GNU Awk, GNU Guile, and Python needed adjustment. Occasionally, occurrences of /-bin/sh are not be handled automatically, for instance in test suites; these have to be patched manually in the package’s recipe.
Bootstrapping
Bootstrapping in our context refers to how the distribution gets built "from nothing". Remember that the build environment of a derivation contains nothing but its declared inputs. So there’s an obvious chicken-and-egg problem: how does the first package get built? How does the first compiler get compiled?
The GNU system we are building is primarily made of C code, with libc at its core. The GNU build system itself assumes the availability of a Bourne shell, traditional Unix tools provided by GNU Coreutils, GNU Awk, Findutils, sed, and grep. Furthermore, our build programs are written in Guile Scheme. Consequently, we rely on pre-built statically-linked binaries of GCC, Binutils, libc, and the other packages mentioned above to get started.
Figure 10 shows the very beginning of the dependency graph of our distribution. At this level of detail, things are slightly more complex. First, Guile itself consists of an ELF executable, along with many source and compiled Scheme files that are dynamically loaded when it runs. This gets stored in the guile-2.0.7.tar.xz tarball shown in this graph. This tarball is part of Guix’s "source" distribution, and gets inserted into the store with add-to-store.
But how do we write a derivation that unpacks this tarball and adds it to the store? To solve this problem, the guile-bootstrap-2.0.drv derivation—the first one that gets built— uses bash as its builder, which runs build-bootstrap-guile.sh, which in turn calls tar to unpack the tarball. Thus, bash, tar, xz, and mkdir are statically-linked binaries, also part of the Guix source distribution, whose sole purpose is to allow the Guile tarball to be unpacked.
Once guile-bootstrap-2.0.drv is built, we have a functioning Guile that can be used to run subsequent build programs. Its first task is to download tarballs containing the other pre-built binaries—this is what the .tar.xz.drv derivations do. Guix modules such as ftp-client.scm are used for this purpose. The module-import.drv derivations import those modules in a directory in the store, using the original layout.[n 1] The module-import-compiled.drv derivations compile those modules, and write them in an output directory with the right layout. This corresponds to the #:module argument of build-expression->derivation mentioned in Section 3.2.
Finally, the various tarballs are unpacked by the derivations gcc-bootstrap-0.drv, glibc-bootstrap-0.drv, etc., at which point we have a working C GNU tool chain. The first tool that gets built with these tools (not shown here) is GNU Make, which is a prerequisite for all the following packages.
Bootstrapping is complete when we have a full tool chain that does not depend on the pre-built bootstrap tools shown in Figure 10. Ways to achieve this are known, and notably
documented by the Linux From Scratch project.[3] We can formally verify this no-dependency requirement by checking whether the files of the final tool chain contain references to the /nix/store directories of the bootstrap inputs.
Obviously, Guix contains package declarations to build the bootstrap binaries shown in Figure 10. Because the final tool chain does not depend on those tools, they rarely need to be updated. Having a way to do that automatically proves to be useful, though. Coupled with Guix’s nascent support for cross-compilation, porting to a new architecture will boil down to cross-building all these bootstrap tools.
RELATED WORK
Numerous package managers for Scheme programs and libraries have been developed, including Racket’s PLaneT, Doro dango for R6RS implementations, Chicken Scheme’s "Eggs", Guildhall for Guile, and ScmPkg.[4] Unlike GNU Guix, they are typically limited to Scheme-only code, and take the core operating system software for granted. To our knowledge, they implement the imperative package management paradigm, and do not attempt to support features such as transactional upgrades and rollbacks. Unsurprisingly, these tools rely on package descriptions that more or less resemble those described in Section 3.3; however, in the case of at least ScmPkg, Doro dango, and Guildhall, package descriptions are written in an external DSL, which happens to use s-expression syntax.
In [21][5], the authors illustrate how the units mechanism of MzScheme modules could be leveraged to improve operating system packaging systems. The examples therein focus on OS services, and multiple instantiation thereof, rather than on package builds and composition.
The Nix package manager is the primary source of inspiration for Guix[6][7]. As noted in Section 2.3, Guix reuses the low-level build and deployment mechanisms of Nix, but differs in its programming interface and preferred implementation language for build scripts. While the Nix language relies on
- ↑ M. Sperber, R. K. Dybvig, M. Flatt, A. V. Straaten, R. B. Findler, J. Matthews. Revised6 Report on the Algorithmic Language Scheme. In Journal of Functional Programming, 19, 7 2009, pp. 1-301.
- ↑ L. Courtès. Down with /bin/sh!. January 2013. https://lists.gnu.org/archive/html/bug-guix/2013-01/msg00041.html.
- ↑ G. Beekmans, M. Burgess, B. Dubbs. Linux From Scratch. 2013. http://www.linuxfromscratch.org/lfs/
- ↑ M. Serrano, E. Gallesio. An Adaptive Package Management System for Scheme. In Proceedings of the 2007 Symposium on Dynamic languages, DLS'07, pp. 65-76, ACM, 2007.
- ↑ D. B. Tucker, S. Krishnamurthi. Applying Module System Research to Package Management. In Proceedings of the Tenth International Workshop on Software Configuration Management, 2001.
- ↑ E. Dolstra, M. d. Jonge, E. Visser. Nix: A Safe and Policy-Free System for Software Deployment. In Proceedings of the 18th Large Installation System Administration Conference (LISA '04), pp. 79-92, USENIX, November 2004.
- ↑ E. Dolstra, A. Löh, N. Pierron. NixOS: A Purely Functional Linux Distribution. In Journal of Functional Programming, (5-6) , New York, NY, USA, November 2010, pp. 577-615.
- ↑ In Guile, module names are a list of symbols, such as (
guix ftp-client), which map directly to file names, such asguix/ftp-client.scm.
