Cross-arch Go tests with QEMU
August 28, 2021

This is handy:

$ GOARCH=arm64 go test
PASS
ok      github.com/cespare/xxhash/v2    0.201s

My machine has an amd64 CPU. How does this work?

For a long time (since Go 1.5), compiling for a different architecture has been as easy as setting the appropriate GOARCH. go test works in two steps: it synthesizes and compiles a main package with all the tests and then runs it. So GOARCH=arm64 go test tells the build step to compile for arm64, not amd64.

Normally that means that this command fails at the run step:

$ GOARCH=arm64 go test
fork/exec /tmp/go-build2883591561/b001/xxhash.test: exec format error
FAIL    github.com/cespare/xxhash/v2    0.001s

However, we can use QEMU to emulate the target architecture. Specifically, we can use QEMU’s user mode emulation to run the test binary directly. In this mode QEMU translates the syscalls and takes care of other details so that the program can run using the host kernel rather than emulating an entire machine.

$ GOARCH=arm64 go test -c -o test.arm64
$ qemu-aarch64 test.arm64
PASS

Then the Linux kernel has a feature called binfmt_misc which allows the user to associate arbitrary executable formats with userspace programs. On Ubuntu, if you install the qemu-user-binfmt package (which comes along as a recommend package if you apt install qemu), it will register all the executable formats QEMU supports using binfmt_misc. Thus:

$ GOARCH=arm64 go test -c -o test.arm64
$ ./test.arm64
PASS

or simply

$ GOARCH=arm64 go test
PASS
ok      github.com/cespare/xxhash/v2    0.191s

I think that’s pretty neat!

Cross-arch Go tests with GitHub Actions

GitHub Actions is a convenient way to run automated tests for projects on GitHub, but the free VMs they provide are currently amd64-only. Hopefully in the future they will add more architectures, but in the meantime, we can use the method described here to run cross-arch tests using QEMU.

In a workflow file, use the docker/setup-qemu-action action to install QEMU static binaries and configure binfmt_misc. Then you can run go test with the appropriate value of GOARCH. The workflow file for github.com/cespare/xxhash is a working example.

Caveats

One limitation here is that it assumes a statically linked binary; if your code requires cgo, you’ll need add -extldflags=-static to the linker flags and use a C compiler for the target architecture. (Also, note that the Go tool sets CGO_ENABLED=0 for cross-compiles by default.)

It should also be possible to make this work with dynamically linked binary. This Debian Wiki page suggests that you’d need to install the libc6 package for the target architecture so that QEMU will have access to the appropriate ELF interpreter (I haven’t tried it).