How to build a multi architecture container using buildx
7 min read | by Jordi Prats
Docker has the ability to handle multi architecture containers: Using the same container image and tag we can deploy it on multiple architectures such as Intel and ARM. Since a docker container is composed of multiple layers it will just use one or another depending on the architecture we are running it. From the user perspective there's no difference on it's usage, but how do we build them?
To be able to create multi architecture containers from a Dockerfile we will have to use docker 19.03+ and enabling the experimental features by editing the /etc/docker/daemon.json as follows:
{
"experimental": true
}
Then we will create a new builder using docker buildx create specifying the list of plaftforms we want to be able to use:
$ docker buildx create --platform linux/amd64,linux/arm64,linux/arm/v7 --use
cranky_ishizaka
We can check the list of available builders using docker buildx ls:
$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS
cranky_ishizaka * docker-container
cranky_ishizaka0 unix:///var/run/docker.sock inactive linux/amd64*, linux/arm64*, linux/arm/v7*
default docker
default default running linux/amd64, linux/386
Then we will have to make sure we have the appropriate emulators correcty installed on the system. The easiest way of doing so is by using tonistiigi/binfmt as follows:
$ docker run --privileged --rm tonistiigi/binfmt --install arm64,arm
Unable to find image 'tonistiigi/binfmt:latest' locally
latest: Pulling from tonistiigi/binfmt
9e0174275344: Pull complete
f163282b5573: Pull complete
Digest: sha256:f52cfb6019e8c8d12b13093fd99c2979ab5631c99f7d8b46d10f899d0d56d6ab
Status: Downloaded newer image for tonistiigi/binfmt:latest
2021/06/09 16:31:01 installing: arm64 OK
2021/06/09 16:31:01 installing: arm OK
{
"supported": [
"linux/amd64",
"linux/arm64",
"linux/386",
"linux/arm/v7",
"linux/arm/v6"
],
"emulators": [
"qemu-aarch64",
"qemu-arm"
]
}
We can check whether it is working by running a container using one of the target architectures, for example, for arm64 we can use arm64v8/alpine:
$ docker run --rm arm64v8/alpine uname -a
Unable to find image 'arm64v8/alpine:latest' locally
latest: Pulling from arm64v8/alpine
595b0fe564bb: Pull complete
Digest: sha256:8f18fae117ec6e5777cc62ba78cbb3be10a8a38639ccfb949521abd95c8301a4
Status: Downloaded newer image for arm64v8/alpine:latest
Linux fa7c06650014 5.8.0-50-generic #56~20.04.1-Ubuntu SMP Mon Apr 12 21:46:35 UTC 2021 aarch64 Linux
Same for armv7:
$ docker run --rm arm32v7/alpine uname -a
Unable to find image 'arm32v7/alpine:latest' locally
latest: Pulling from arm32v7/alpine
e160e00eb35d: Pull complete
Digest: sha256:9663906b1c3bf891618ebcac857961531357525b25493ef717bca0f86f581ad6
Status: Downloaded newer image for arm32v7/alpine:latest
Linux 2975b7153aa0 5.8.0-50-generic #56~20.04.1-Ubuntu SMP Mon Apr 12 21:46:35 UTC 2021 armv7l Linux
To be able to build a multi architecture container from a Dockerfile the command is not much different than the one we are used to: We just need to use buildx by invoking it using docker buildx build and then set the target platforms we want to use with the --platform option.
We have several output options that we can select using the -o type= option. The registry option is handy since it's going to push it directly into the registry.
Oddly enough, the build status appears on the top of the output instead of at the end:
$ docker buildx build --platform linux/arm/v7,linux/arm64,linux/amd64 -t jordiprats/pet2cattle-mutating-webhook:1 -o type=registry .
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 48.9s (25/25) FINISHED
=> [internal] load build definition from Dockerfile 2.8s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 2.9s
=> => transferring context: 2B 0.0s
=> [linux/arm/v7 internal] load metadata for docker.io/library/python:3.8-alpine 1.7s
=> [linux/amd64 internal] load metadata for docker.io/library/python:3.8-alpine 1.7s
=> [linux/arm64 internal] load metadata for docker.io/library/python:3.8-alpine 1.7s
=> [auth] library/python:pull token for registry-1.docker.io 0.0s
=> [internal] load build context 0.3s
=> => transferring context: 280B 0.0s
=> [linux/amd64 1/6] FROM docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 0.8s
=> => resolve docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 0.7s
=> [linux/arm/v7 1/6] FROM docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 1.3s
=> => resolve docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 1.2s
=> [linux/arm64 1/6] FROM docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 1.7s
=> => resolve docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 1.6s
=> CACHED [linux/amd64 2/6] WORKDIR /code 0.0s
=> CACHED [linux/amd64 3/6] RUN pip install gunicorn 0.0s
=> CACHED [linux/amd64 4/6] COPY requirements.txt . 0.0s
=> CACHED [linux/amd64 5/6] RUN pip install -r requirements.txt 0.0s
=> CACHED [linux/amd64 6/6] COPY app /code/app 0.0s
=> CACHED [linux/arm/v7 2/6] WORKDIR /code 0.0s
=> [linux/arm/v7 3/6] RUN pip install gunicorn 14.5s
=> CACHED [linux/arm64 2/6] WORKDIR /code 0.0s
=> [linux/arm64 3/6] RUN pip install gunicorn 14.2s
=> [linux/arm/v7 4/6] COPY requirements.txt . 3.6s
=> [linux/arm64 4/6] COPY requirements.txt . 4.0s
=> [linux/arm/v7 5/6] RUN pip install -r requirements.txt 20.0s
=> [linux/arm64 5/6] RUN pip install -r requirements.txt 22.7s
=> [linux/arm/v7 6/6] COPY app /code/app 1.7s
=> [linux/arm64 6/6] COPY app /code/app 1.2s
If there's something wrong with the setup we can end up with some nasty results that makes it hard to understand what's going on, for example:
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c pip install gunicorn]: exit code: 1
Doesn't tell much about what's going on, right? For this specific message, what's going on is that the underlying qemu is not setup correctly, so we can fix it using tonistiigi/binfmt. The full output is:
$ docker buildx build --platform linux/arm/v7,linux/arm64,linux/amd64 -t test .
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 20.4s (18/24)
=> [internal] load build definition from Dockerfile 2.1s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 2.2s
=> => transferring context: 2B 0.0s
=> [linux/amd64 internal] load metadata for docker.io/library/python:3.8-alpine 0.9s
=> [linux/arm/v7 internal] load metadata for docker.io/library/python:3.8-alpine 3.1s
=> [linux/arm64 internal] load metadata for docker.io/library/python:3.8-alpine 0.9s
=> [internal] load build context 1.4s
=> => transferring context: 280B 0.0s
=> [linux/arm64 1/6] FROM docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 2.3s
=> => resolve docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 1.1s
=> [linux/arm/v7 1/6] FROM docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 2.3s
=> => resolve docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 1.1s
=> [linux/amd64 1/6] FROM docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 2.4s
=> => resolve docker.io/library/python:3.8-alpine@sha256:9e296fa6f65f947a62398b32b5a54fdfd3f04f3a2c58584172bbe988f794a898 1.1s
=> CACHED [linux/arm64 2/6] WORKDIR /code 0.0s
=> ERROR [linux/arm64 3/6] RUN pip install gunicorn 12.5s
=> CACHED [linux/arm/v7 2/6] WORKDIR /code 0.0s
=> ERROR [linux/arm/v7 3/6] RUN pip install gunicorn 12.6s
=> CACHED [linux/amd64 2/6] WORKDIR /code 0.0s
=> CACHED [linux/amd64 3/6] RUN pip install gunicorn 0.0s
=> CACHED [linux/amd64 4/6] COPY requirements.txt . 0.0s
=> CACHED [linux/amd64 5/6] RUN pip install -r requirements.txt 0.0s
=> CACHED [linux/amd64 6/6] COPY app /code/app 0.0s
------
> [linux/arm64 3/6] RUN pip install gunicorn:
#8 7.794 ERROR: Exception:
#8 7.794 Traceback (most recent call last):
#8 7.794 File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
(...)
#8 7.794 subprocess.CalledProcessError: Command '('lsb_release', '-a')' returned non-zero exit status 1.
#8 7.814 Traceback (most recent call last):
#8 7.814 File "/usr/local/bin/pip", line 8, in <module>
(...)
#8 7.814 subprocess.CalledProcessError: Command '('lsb_release', '-a')' returned non-zero exit status 1.
------
------
> [linux/arm/v7 3/6] RUN pip install gunicorn:
#21 9.954 ERROR: Exception:
#21 9.954 Traceback (most recent call last):
(...)
#21 9.954 raise CalledProcessError(retcode, process.args,
#21 9.954 subprocess.CalledProcessError: Command '('lsb_release', '-a')' returned non-zero exit status 1.
#21 9.975 Traceback (most recent call last):
(...)
#21 9.975 File "/usr/local/lib/python3.8/subprocess.py", line 516, in run
#21 9.975 raise CalledProcessError(retcode, process.args,
#21 9.975 subprocess.CalledProcessError: Command '('lsb_release', '-a')' returned non-zero exit status 1.
------
Dockerfile:6
--------------------
4 |
5 | # GUNICORN - not an actual dependency
6 | >>> RUN pip install gunicorn
7 |
8 | COPY requirements.txt .
--------------------
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c pip install gunicorn]: exit code: 1
Posted on 10/06/2021