K3s installation on a Raspberry Pi

4 min read | by Jordi Prats

To install k3s on a Raspberry Pi looks quite straight forward but it has it's nuisances.

Installing k3s should be a single line command:

# curl -sfL https://get.k3s.io | sh -
[INFO]  Finding release for channel stable
[INFO]  Using v1.18.9+k3s1 as release
[INFO]  Downloading hash https://github.com/rancher/k3s/releases/download/v1.18.9+k3s1/sha256sum-arm.txt
[INFO]  Downloading binary https://github.com/rancher/k3s/releases/download/v1.18.9+k3s1/k3s-armhf
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s

So far looks like everything should be just fine, but if you check the k3s status using systemctl you will realize it refuses to start:

# systemctl status k3s
● k3s.service - Lightweight Kubernetes
   Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Tue 2020-10-13 19:38:00 BST; 4s ago
     Docs: https://k3s.io
  Process: 2107 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
  Process: 2108 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
  Process: 2109 ExecStart=/usr/local/bin/k3s server (code=exited, status=1/FAILURE)
 Main PID: 2109 (code=exited, status=1/FAILURE)

You will have to start k3s manually to be able to get a helpful error message:

# /usr/local/bin/k3s server --no-deploy servicelb --no-deploy traefik
ERRO[2020-10-13T19:40:58.690824994+01:00] Failed to find memory cgroup, you may need to add "cgroup_memory=1 cgroup_enable=memory" to your linux cmdline (/boot/cmdline.txt on a Raspberry Pi) 
FATA[2020-10-13T19:40:58.691126492+01:00] failed to find memory cgroup, you may need to add "cgroup_memory=1 cgroup_enable=memory" to your linux cmdline (/boot/cmdline.txt on a Raspberry Pi)

You can check that memory cgroups are disabled by looking for "memory" on /proc/cgroups:

$ cat /proc/cgroups 
#subsys_name  hierarchy num_cgroups enabled
(...)
memory  0 87  0
(...)

The last column tells us whether it is enabled or not. We will have to add "cgroup_memory=1 cgroup_enable=memory" to the cmdline.txt and reboot the system.

Afterwards, we can check again this file to make sure everything if good to go:

$ cat memory /proc/cgroups 
#subsys_name  hierarchy num_cgroups enabled
(...)
memory  8 186 1
(...)

Once enabled, k3s will start on boot:

$ kubectl get nodes
NAME                                 STATUS   ROLES    AGE   VERSION
datalore.lolcathost.systemadmin.es   Ready    master   59d   v1.18.9+k3s1

Update: A while ago, I created a pull request (PR-2379) to k3s adding an installation hint for the issue explained on this post, since it got merged now you'll see the following instead:

root@rpi-n01:~# curl -sfL https://get.k3s.io | sh -
[INFO]  Finding release for channel stable
[INFO]  Using v1.21.4+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.21.4+k3s1/sha256sum-arm64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.21.4+k3s1/k3s-arm64
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Creating /usr/local/bin/ctr symlink to k3s
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  Failed to find memory cgroup, you may need to add "cgroup_memory=1 cgroup_enable=memory" to your linux cmdline (/boot/cmdline.txt on a Raspberry Pi)
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s
Job for k3s.service failed because the control process exited with error code.
See "systemctl status k3s.service" and "journalctl -xe" for details.

Posted on 20/12/2020

K3s installation on a Raspberry Pi

Categories