Install AWS cloud provider on a k3s cluster

k3s aws cloud provider load balancer

2 min read | by Jordi Prats

One of the drawbacks of installing k3s on a EC2 instance versus using EKS is that we loose the AWS integration, so we cannot use AWS load balancers by default. Thanks to the AWS cloud provider we can overcome this limitation

The AWS cloud provider provides the interface between the Kubernetes cluster and AWS, being able to provision, monitor and remove AWS resources necessary for operation of the cluster.

With this controller we will be able to create and update AWS load balancers (classic and NLB) and handle the node lifecycle. If we want to create Volumes or handle the creation of ALB load balancers we will have to add additional controllers

To be able to use the AWS cloud provider we'll need to disable the default cloud controller that comes with K3S together with servicelb and traefik since we are going to use AWS services that provide the same functionality:

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" sh -s - \
  --disable-cloud-controller" --disable servicelb" --disable traefik"

We will also need to make sure the instance has enough privileges via it's instance profile, on the AWS cloud provider we can find the IAM policies that it requires

To make sure this helm chart is installed on the k3s cluster we can create a HelmChart definition on /var/lib/rancher/k3s/server/manifests as follows:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: aws-cloud-controller-manager
  namespace: kube-system
spec:
  chart: https://github.com/kubernetes/cloud-provider-aws/releases/download/helm-chart-aws-cloud-controller-manager-0.0.6/aws-cloud-controller-manager-0.0.6.tgz
  targetNamespace: kube-system
  bootstrap: true
  valuesContent: |-
    hostNetworking: true
    nodeSelector:
      node-role.kubernetes.io/master: "true"

k3s is going to make sure this object stays in sync with the cluster, installing it if needed

Posted on 10/05/2022

Categories