AWS EKS: How to resize Persistent Volumes

3 min read

On our AWS EKS cluster, at the time of this writing, by default you cannot resize volumes provisioned with the default gp2 StorageClass, allowVolumeExpansion is set to false:

$ kubectl get sc
NAME            PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
gp2 (default)   kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  78d

By default this setting is not present on the StorageClass so we can simply add to the existing definition like this:

$ echo -e "$(kubectl get sc gp2 -o yaml)\nallowVolumeExpansion: true" | kubectl apply -f -
storageclass.storage.k8s.io/gp2 configured

This setting tells the Kubernetes CSI that the underlying volume, in this case an EBS Volume, can be resized.

Using kubectl get sc we can see how it changed to ALLOWVOLUMEEXPANSION true:

$ kubectl get sc 
NAME            PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
gp2 (default)   kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   true                   78d

Let's check the size of the voluem by using kubectl get pv or by checking the volume size on the pod:

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                          STORAGECLASS   REASON   AGE
pvc-a1448f38-5f28-492e-a09c-8a900b9fb43e   35Gi       RWO            Delete           Bound    pet2cattle/pet2cattle-static   gp2                     9d6h
$ kubectl exec -it pet2cattle-79979695b-7rmg6 -- df -hP
Filesystem      Size  Used Avail Use% Mounted on
overlay          20G   11G  9.5G  53% /
tmpfs            64M     0   64M   0% /dev
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/xvda1       20G   11G  9.5G  53% /tmp
shm              64M     0   64M   0% /dev/shm
/dev/xvdbg       35G  1.3G   34G   4% /opt/pet2cattle/static
tmpfs           3.9G   12K  3.9G   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs           3.9G     0  3.9G   0% /proc/acpi
tmpfs           3.9G     0  3.9G   0% /proc/scsi
tmpfs           3.9G     0  3.9G   0% /sys/firmware

We can resize the volume by setting the storage property to the size we would like:

$ kubectl get pvc pet2cattle-data -o yaml | sed 's/storage: 35Gi/storage: 40Gi/g' | kubectl apply -f -
persistentvolumeclaim/pet2cattle-data configured

After a while we will be able to see how the volume have been resized:

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                          STORAGECLASS   REASON   AGE
pvc-a1448f38-5f28-492e-a09c-8a900b9fb43e   40Gi       RWO            Delete           Bound    pet2cattle/pet2cattle-static   gp2                     9d6h
$ kubectl exec -it pet2cattle-79979695b-7rmg6 -- df -hP
Filesystem      Size  Used Avail Use% Mounted on
overlay          20G   11G  9.5G  53% /
tmpfs            64M     0   64M   0% /dev
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/xvda1       20G   11G  9.5G  53% /tmp
shm              64M     0   64M   0% /dev/shm
/dev/xvdbg       40G  1.3G   38G   4% /opt/pet2cattle/static
tmpfs           3.9G   12K  3.9G   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs           3.9G     0  3.9G   0% /proc/acpi
tmpfs           3.9G     0  3.9G   0% /proc/scsi
tmpfs           3.9G     0  3.9G   0% /sys/firmware

Posted on 10/05/2021

Categories