Facebook Linkedin Twitter
Posted Mon Dec 20, 2021 •  Reading time: 4 minutes

How chezmoi Implements Cross-Platform CI

Go makes cross-compilation easy, but how do you run cross-platform tests?

chezmoi is a popular cross-platform dotfile manager written in Go, with particularly strong support for managing multiple diverse machines and security.

One of chezmoi’s goals is to support multiple operating systems. It’s one of only a handful of dotfile managers that support Windows, FreeBSD, OpenBSD, and Illumos, as well as the usual stalwarts of macOS and Linux, all as first-class citizens. For these systems to be well-supported, it is essential to test on them.

Go is an excellent language for writing cross-platform command line tools, thanks to easy cross-compilation, a wide range of supported operating systems and architectures, and the simplicity of distributing a single statically-linked binary.

chezmoi uses GitHub, GitHub Actions, and GoReleaser in its continuous integration pipelines to build, test, and distribute Go binaries for twenty four operating system/architecture combinations. This rich pipeline, combined with an extensive suite of unit and integration tests (using the excellent testscript package), ensures that many bugs are caught early while they are cheap to fix, and well before they reach users.

Cross-Linux distribution testing with Docker

Linux distributions vary significantly, and if you want to ensure that your software runs correctly then testing is imperative. Any code that is not tested is probably broken. Significant variations across Linux distributions that impact Go programs include the libc (typically glibc or musl), the default umask, and the location of system configuration files.

chezmoi uses Docker to quickly test multiple Linux distributions on a single Linux host. The strategy is roughly:

  1. Build a docker image for the Linux distribution under test including Go.
  2. Mount our source code in the docker container.
  3. Run go test in the docker container.

For example, the Dockerfile for Alpine Linux (a musl-based distribution), contains effectively:

FROM alpine:latest

# install build and integration test dependencies
RUN apk add --no-cache age git go unzip zip

ENTRYPOINT ( cd /src && go test ./... )

Build the container and run the tests with two lines of shell:

$ image="$(docker build . -q)"
$ docker run --rm --volume "${PWD}:/src" "${image}"

Running this as a continuous integration test in GitHub Actions is a single job:

jobs:
  test-alpine:
    runs-on: ubuntu-20.04
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    - name: Test
      run: |
        image="$(docker build . -q)"
        docker run --rm --volume "${PWD}:/src" "${image}"        

In practice, Docker prefers one Dockerfile per directory, so you need a few extra flags and changes of directory to support testing on multiple Linux distributions. For a concrete example, see chezmoi’s Dockerfiles for Alpine Linux, Arch Linux, Fedora, and Void Linux, an example corresponding GitHub Action job, and a bash script that ties everything together.

Cross-operating system testing with GitHub Actions

GitHub Action runners support Linux, macOS, and Windows, so we can test these operating systems directly, although this currently limited to the amd64 architecture only.

As this is typical use of GitHub Actions, we’ll skip the details here, but you can browse chezmoi’s job for testing on Windows, which includes installing integration test dependencies with Chocolately.

Cross-operating system testing with Vagrant

So how do we test on other operating systems and architectures, where GitHub Action runners are not available?

Vagrant makes it easy to spin up virtual machines, and, luckily for us, GitHub’s macos-10.15 runners include Vagrant. So, we adopt a similar strategy to the cross-Linux distribution tests:

  1. Build a virtual machine image for the system under test, including Go.
  2. Copy our source code in to the image.
  3. Run go test in the virtual machine.

Example Vagrantfiless for Debian (i386 architecture), FreeBSD, OpenBSD, and OpenIndiana can be found in chezmoi’s GitHub repo, and install Go and any integration test dependencies using the system’s package manager. For example, the Vagrantfile for FreeBSD contains:

Vagrant.configure("2") do |config|
  config.vm.box = "generic/freebsd13"
  config.vm.define :freebsd13
  config.vm.hostname = "freebsd13"
  config.vm.synced_folder ".", "/chezmoi", type: "rsync"
  config.vm.provision "shell", inline: <<-SHELL
    pkg install --quiet --yes age git gnupg go zip
  SHELL
  config.vm.provision "file", source: "assets/vagrant/freebsd13.test-chezmoi.sh", destination: "test-chezmoi.sh"
end

Vagrant virtual machine images can be large (easily several hundred megabytes) and it is important that we cache them in GitHub Actions, otherwise Vagrant will quickly rate limit us. actions/cache can do this. At the time of writing, the GitHub Actions cache is limited to 5GB, so you can cache a few virtual machine images. Here’s the full job for testing chezmoi on FreeBSD:

jobs:
  test-freebsd:
    runs-on: macos-10.15
    env:
      VAGRANT_BOX: freebsd13
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    - name: Cache Vagrant Boxes
      uses: actions/cache@v2
      with:
        path: ~/.vagrant.d
        key: ${{ runner.os }}-vagrant-freebsd13-${{ hashFiles('assets/vagrant/freebsd13.Vagrantfile') }}
        restore-keys: |
                    ${{ runner.os }}-vagrant-freebsd13-
    - name: Test
      run: |
                ( cd assets/vagrant && ./test.sh freebsd13 )

actions/vagrant/test.sh is a short shell script to bring the box up, run the tests, and destroy the box:

#!/bin/bash

set -eufo pipefail

os=$1
export VAGRANT_VAGRANTFILE=assets/vagrant/${os}.Vagrantfile
if ! ( cd ../.. && vagrant up ); then
    exit 1
fi
vagrant ssh -c "./test-chezmoi.sh"
vagrant_ssh_exit_code=$?
vagrant destroy -f || exit 1
exit $vagrant_ssh_exit_code

As with Docker, Vagrant prefers one Vagrantfile per directory, so a bash script is needed to tie everything together.

Cross-platform packaging with GoReleaser

Of course, cross-operating system testing is only one step along the way of getting your software into the hands of users. chezmoi uses the excellent GoReleaser project to build binaries, tarballs, packages, changelogs, GitHub releases, and more, all triggered by simply pushing a tag to a GitHub repo, but that is the subject for another article.

Conclusion

In this article we learned how to test Go command line applications across multiple Linux distributions, operating systems and architectures:

  1. Docker provides a fast way to test across multiple Linux distributions.
  2. GitHub Actions provides a fast way to test on macOS, Linux, and Windows.
  3. Vagrant on GitHub Actions provides a way to test across other operating systems and architectures.