How chezmoi Implements Cross-Platform CI
Go makes cross-compilation easy, but how do you run cross-platform tests?
chezmoi is a popular cross-platform dotfile manager written in Go, with particularly strong support for managing multiple diverse machines and security.
One of chezmoi’s goals is to support multiple operating systems. It’s one of only a handful of dotfile managers that support Windows, FreeBSD, OpenBSD, and Illumos, as well as the usual stalwarts of macOS and Linux, all as first-class citizens. For these systems to be well-supported, it is essential to test on them.
Go is an excellent language for writing cross-platform command line tools, thanks to easy cross-compilation, a wide range of supported operating systems and architectures, and the simplicity of distributing a single statically-linked binary.
chezmoi uses GitHub, GitHub Actions, and GoReleaser in its continuous integration pipelines to build, test, and distribute Go binaries for twenty four operating system/architecture combinations. This rich pipeline, combined with an extensive suite of unit and integration tests (using the excellent testscript package), ensures that many bugs are caught early while they are cheap to fix, and well before they reach users.
Cross-Linux distribution testing with Docker
Linux distributions vary significantly, and if you want to ensure that your software runs correctly then testing is imperative. Any code that is not tested is probably broken. Significant variations across Linux distributions that impact Go programs include the libc (typically glibc or musl), the default umask, and the location of system configuration files.
chezmoi uses Docker to quickly test multiple Linux distributions on a single Linux host. The strategy is roughly:
- Build a docker image for the Linux distribution under test including Go.
- Mount our source code in the docker container.
- Run
go test
in the docker container.
For example, the Dockerfile
for Alpine Linux (a musl-based distribution),
contains effectively:
FROM alpine:latest
# install build and integration test dependencies
RUN apk add --no-cache age git go unzip zip
ENTRYPOINT ( cd /src && go test ./... )
Build the container and run the tests with two lines of shell:
$ image="$(docker build . -q)"
$ docker run --rm --volume "${PWD}:/src" "${image}"
Running this as a continuous integration test in GitHub Actions is a single job:
jobs:
test-alpine:
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Test
run: |
image="$(docker build . -q)"
docker run --rm --volume "${PWD}:/src" "${image}"
In practice, Docker prefers one Dockerfile
per directory, so you need a few
extra flags and changes of directory to support testing on multiple Linux
distributions. For a concrete example, see chezmoi’s Dockerfile
s for Alpine
Linux, Arch Linux, Fedora, and Void
Linux, an example
corresponding GitHub Action
job,
and a bash script that ties everything
together.
Cross-operating system testing with GitHub Actions
GitHub Action runners support Linux, macOS, and Windows, so we can test these
operating systems directly, although this currently limited to the amd64
architecture only.
As this is typical use of GitHub Actions, we’ll skip the details here, but you can browse chezmoi’s job for testing on Windows, which includes installing integration test dependencies with Chocolately.
Cross-operating system testing with Vagrant
So how do we test on other operating systems and architectures, where GitHub Action runners are not available?
Vagrant makes it easy to spin up virtual machines,
and, luckily for us, GitHub’s macos-10.15
runners include Vagrant. So, we
adopt a similar strategy to the cross-Linux distribution tests:
- Build a virtual machine image for the system under test, including Go.
- Copy our source code in to the image.
- Run
go test
in the virtual machine.
Example Vagrantfiles
s for Debian (i386 architecture), FreeBSD, OpenBSD, and
OpenIndiana can be found in chezmoi’s GitHub
repo, and
install Go and any integration test dependencies using the system’s package
manager. For example, the Vagrantfile
for FreeBSD contains:
Vagrant.configure("2") do |config|
config.vm.box = "generic/freebsd13"
config.vm.define :freebsd13
config.vm.hostname = "freebsd13"
config.vm.synced_folder ".", "/chezmoi", type: "rsync"
config.vm.provision "shell", inline: <<-SHELL
pkg install --quiet --yes age git gnupg go zip
SHELL
config.vm.provision "file", source: "assets/vagrant/freebsd13.test-chezmoi.sh", destination: "test-chezmoi.sh"
end
Vagrant virtual machine images can be large (easily several hundred megabytes)
and it is important that we cache them in GitHub Actions, otherwise Vagrant will
quickly rate limit us. actions/cache
can
do this. At the time of writing, the GitHub Actions cache is limited to 5GB, so
you can cache a few virtual machine images. Here’s the full job for testing
chezmoi on FreeBSD:
jobs:
test-freebsd:
runs-on: macos-10.15
env:
VAGRANT_BOX: freebsd13
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Cache Vagrant Boxes
uses: actions/cache@v2
with:
path: ~/.vagrant.d
key: ${{ runner.os }}-vagrant-freebsd13-${{ hashFiles('assets/vagrant/freebsd13.Vagrantfile') }}
restore-keys: |
${{ runner.os }}-vagrant-freebsd13-
- name: Test
run: |
( cd assets/vagrant && ./test.sh freebsd13 )
actions/vagrant/test.sh
is a short shell script to bring the box up, run the
tests, and destroy the box:
#!/bin/bash
set -eufo pipefail
os=$1
export VAGRANT_VAGRANTFILE=assets/vagrant/${os}.Vagrantfile
if ! ( cd ../.. && vagrant up ); then
exit 1
fi
vagrant ssh -c "./test-chezmoi.sh"
vagrant_ssh_exit_code=$?
vagrant destroy -f || exit 1
exit $vagrant_ssh_exit_code
As with Docker, Vagrant prefers one Vagrantfile
per directory, so a bash
script
is needed to tie everything together.
Cross-platform packaging with GoReleaser
Of course, cross-operating system testing is only one step along the way of getting your software into the hands of users. chezmoi uses the excellent GoReleaser project to build binaries, tarballs, packages, changelogs, GitHub releases, and more, all triggered by simply pushing a tag to a GitHub repo, but that is the subject for another article.
Conclusion
In this article we learned how to test Go command line applications across multiple Linux distributions, operating systems and architectures:
- Docker provides a fast way to test across multiple Linux distributions.
- GitHub Actions provides a fast way to test on macOS, Linux, and Windows.
- Vagrant on GitHub Actions provides a way to test across other operating systems and architectures.