Docker engine and Docker CLI on riscv64

mackenzy · March 21, 2022, 8:43pm

Hi,

I was looking for a way to run a current version of Docker on my VisionFive board. As there are no pre-built packages yet (compared to Podman, which is already available via package manager) I’d need to build it from source.
Luckily I stumbled across build instructions dated last year but they are more or less still up to date:

github.com

carlosedp/riscv-bringup/blob/master/build-docker-env.md

# Build requirements for Containers on Risc-V

This doc allows one to build and set the requirements to run containers on Risc-V architecture.

You need a recent working Golang version (1.16) installed and in the path. Check releases to download a prebuilt package.

It has been tested on a SiFive Unleashed board.

Create a temporary place for building:

```bash
mkdir -p $HOME/riscv-docker
cd $HOME/riscv-docker
```

If building Debian `.deb`, create a destination dir:

```bash
mkdir -p $HOME/riscv-docker/debs
```

This file has been truncated. show original

I’ve been using a Lubuntu 21.10 image which is discussed in another thread: Lubuntu 21.10 image for Starfive VisionFive V1

As the flashed image only has a 10gb root partition I had to resize it otherwise I’d have ended in a no space left error.

Package-wise I’ve only installed the following:

sudo apt install build-essential cmake pkg-config libseccomp2 libseccomp-dev libdevmapper-dev golang libbtrfs-dev

Not quite sure which Go version is required as a minimum. Likely 16. I’ve used 17 and had no issues.

Steps to build all the necessary binaries, engine and CLI version 20.10.13 (don’t forget to set GOPATH first!):

mkdir -p $HOME/riscv-docker
cd $HOME/riscv-docker

#runc
git clone https://github.com/opencontainers/runc
pushd runc
make
sudo make install
popd

#containerd
git clone https://github.com/containerd/containerd
pushd containerd
make BUILDTAGS="no_btrfs"
sudo make install
popd

#docker-cli
mkdir -p $GOPATH/src/github.com/docker/
pushd $GOPATH/src/github.com/docker/
git clone https://github.com/docker/cli
pushd cli
git checkout v20.10.13
VERSION=20.10.13 DISABLE_WARN_OUTSIDE_CONTAINER=1 GO111MODULE=off GOOS=linux GOARCH=riscv64 make
sudo cp build/docker-linux-riscv64 /usr/local/bin
sudo ln -sf /usr/local/bin/docker-linux-riscv64 /usr/local/bin/docker
popd
popd

#docker-init
git clone https://github.com/krallin/tini
pushd tini
export CFLAGS="-DPR_SET_CHILD_SUBREAPER=36 -DPR_GET_CHILD_SUBREAPER=37"
cmake . && make
sudo cp tini-static /usr/local/bin/docker-init
popd

#docker-proxy
mkdir -p $GOPATH/src/github.com/docker
pushd $GOPATH/src/github.com/docker
git clone https://github.com/docker/libnetwork/
pushd libnetwork
go get github.com/ishidawataru/sctp
GO111MODULE=off go build ./cmd/proxy
sudo cp proxy /usr/local/bin/docker-proxy
popd
popd

#rootlesskit
mkdir -p $GOPATH/src/github.com/rootless-containers/
pushd $GOPATH/src/github.com/rootless-containers/
git clone https://github.com/rootless-containers/rootlesskit.git
pushd rootlesskit
make
sudo make install
popd
popd

#dockerd
mkdir -p $GOPATH/src/github.com/docker/
pushd $GOPATH/src/github.com/docker/
git clone https://github.com/moby/moby docker
pushd docker
git checkout v20.10.13
sudo cp contrib/dockerd-rootless.sh /usr/local/bin
VERSION=20.10.13 DISABLE_WARN_OUTSIDE_CONTAINER=1 GO111MODULE=off GOOS=linux GOARCH=riscv64 ./hack/make.sh binary
sudo cp bundles/binary-daemon/dockerd-20.10.13 /usr/local/bin/dockerd
popd
popd

Took about an hour or more to compile all binaries. Systemd service files can be created as originally stated in the linked how-to.
One of the lasts steps would be to create a docker group, otherwise one of the systemd services did not start.

If nothing went wrong you’re greeted with this:

$ docker version
Client:
 Version:           20.10.13
 API version:       1.41
 Go version:        go1.17
 Git commit:        a224086349
 Built:             Sun Mar 20 18:35:08 2022
 OS/Arch:           linux/riscv64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.13
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17
  Git commit:       906f57ff5b
  Built:            Sun Mar 20 18:13:55 2022
  OS/Arch:          linux/riscv64
  Experimental:     false
 containerd:
  Version:          v1.6.0-117-ged1a76270
  GitCommit:        ed1a762707cea5055dec47ed4b049159ac9f23a3
 runc:
  Version:          1.1.0+dev
  GitCommit:        v1.1.0-92-g98b75bef
 docker-init:
  Version:          0.19.0
  GitCommit:        378bbbc

kth5 · March 21, 2022, 9:29pm

I understand the want to use Docker but many distributions come with systemd and thus a built-in container engine aka systemd-nspawn. Considering how many images you’ll find on Docker Hub working for anything but x86_64, you may as well use a native solution (no static blob built by Go even) should you find the need to torture yourself.

A quick rundown on how things work:

Salem · March 21, 2022, 10:20pm

You can use the following project which will make docker images for other archs work inside RISC-V:

I will test it today inshallah

kth5 · March 21, 2022, 10:33pm

That will emulate the platform the image was made for, this means you’ll have to accept slow TCG by Qemu which is usually running at a fraction of the speed of a similarly clocked native-CPU/core.

Forking is also incredibly slow in user-mode via binfmt, since it’ll have to fork an entire Qemu each time. Imagine a autoconf configure script, nuts when each bash call takes >2s.

It’ll work but it’s hardly a solution for everyone.

mackenzy · March 22, 2022, 12:04pm

There is clearly always another approach also on how to get things done In my use case I just wanted to check out the current status of Docker specifically and also to incorporate it into my zoo of different architectures I’m currently running.

But thanks for highlighting systemd-nspawn, looks quite interesting though

Salem · March 25, 2022, 12:12pm

NVM, You already said that