How to set filesystem permissions on Volumes for non-root containers

3 min read | by Jordi Prats

As a best practice we should try run containers with the minimum privileges they require: If we want to run a container with a non-root user we need to specify the user we want to use with securityContext.runAsUser (unless the container is not already using a non-privileged user).

By doing so when working with Volumes we might get a Permission denied while accessing the container

We can reproduce this problem by running the following StatefulSet that will use a PersistentVolume for /test:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: sts-test-volume-runasuser
spec:
  serviceName: default
  replicas: 1
  selector:
    matchLabels:
      component: test-volume
  volumeClaimTemplates:
  - metadata:
      name: test-volume
      labels:
        component: test-volume
    spec:
      accessModes: [ "ReadWriteOnce" ]
      volumeMode: Filesystem
      resources:
        requests:
          storage: 10Gi
  template:
    metadata:
      labels:
        component: test-volume
    spec:
      securityContext:
        runAsUser: 1000
      containers:
      - name: test-container
        image: "alpine:latest"
        command:
        - sleep
        - 24h
        volumeMounts:
          - mountPath: /test
            name: test-volume

You can find all the yaml files on the pet2cattle/kubernetes-volume-fsgroup repository on GitHub.

If we try to create a file on the Volume we are going to get a Permission denied error:

$ kubectl exec -it sts-test-volume-runasuser-0 -n test -- touch /test/permissions
touch: /test/permissions: Permission denied
command terminated with exit code 1

Because the Volume is mounted as root

$ kubectl exec -it sts-test-volume-runasuser-0 -n test -- ls -ld /test
drwxr-xr-x    3 root     root          4096 Feb 18 04:48 /test

To be able to access the volume we'll need to use the securityContext.fsGroup. By using it, all processes of the container are also part of the supplementary group we set (So it doesn't need to be the same as runAsGroup)

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: sts-test-volume-runasuser-fsgroup
spec:
  serviceName: default
  replicas: 1
  selector:
    matchLabels:
      component: test-volume
  volumeClaimTemplates:
  - metadata:
      name: test-volume
      labels:
        component: test-volume
    spec:
      accessModes: [ "ReadWriteOnce" ]
      volumeMode: Filesystem
      resources:
        requests:
          storage: 10Gi
  template:
    metadata:
      labels:
        component: test-volume
    spec:
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
      containers:
      - name: test-container
        image: "alpine:latest"
        command:
        - sleep
        - 24h
        volumeMounts:
          - mountPath: /test
            name: test-volume

Now we are going to be allowed to create a file on the Volume:

$ kubectl exec -it sts-test-volume-runasuser-fsgroup-0 -n test -- touch /test/permissions
$ kubectl exec -it sts-test-volume-runasuser-fsgroup-0 -n test -- ls -l /test/permissions
-rw-r--r--    1 1000     1000             0 Feb 18 04:50 /test/permissions

We can check how the /test Volume group's is the to securityContext.fsGroup:

$ kubectl exec -it sts-test-volume-runasuser-fsgroup-0 -n test -- ls -ld /test
drwxrwsr-x    3 root     1000          4096 Feb 18 04:50 /test

Posted on 18/02/2022

Categories