Error syncing load balancer: failed to ensure load balancer: could not find any suitable subnets for creating the ELB

2 min read | by Jordi Prats

If we try to create a LoadBalancer on an AWS EKS cluster without any public subnet it will get stuck on the pending state and we won't get any external IP/DNS name for it. By using kubectl describe we will be able to get the actual error:

$ kubectl get svc -n pet2cattle
NAME      TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
demo-lb   LoadBalancer   172.20.235.213   <pending>     80:30525/TCP   7d
$ kubectl describe svc demo-lb -n pet2cattle 
Name:                     demo-lb
Namespace:                pet2cattle
Labels:                   <none>
Annotations:              <none>
Selector:                 run=demo-lb
Type:                     LoadBalancer
IP Families:              <none>
IP:                       172.20.166.181
IPs:                      <none>
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  30088/TCP
Endpoints:                10.236.124.69:80,10.236.126.253:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type     Reason                  Age                From                Message
  ----     ------                  ----               ----                -------
  Normal   EnsuringLoadBalancer    12s (x3 over 27s)  service-controller  Ensuring load balancer
  Warning  SyncLoadBalancerFailed  12s (x3 over 27s)  service-controller  Error syncing load balancer: failed to ensure load balancer: could not find any suitable subnets for creating the ELB

It tries to find a public subnet to expose the service, if we just don't have any it will never provision a LoadBalancer. Since we might want to expose it to the internal subnet we can force to provision the LoadBalancer on the private subnet by adding an annotation to the Service:

service.beta.kubernetes.io/aws-load-balancer-internal: "true"

We can either set it using kubectl annotate:

$ kubectl annotate svc demo-lb -n pet2cattle "service.beta.kubernetes.io/aws-load-balancer-internal"="true"
service/demo-lb annotated

Or by editing the yaml:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"
  name: demo-lb
  namespace: pet2cattle
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: demo-lb
  type: LoadBalancer

After setting the annotation, it will take a while to get the external IP, but eventually it will show up on kube get svc:

$ kubectl get svc 
NAME      TYPE           CLUSTER-IP       EXTERNAL-IP                                                                       PORT(S)        AGE
demo-lb   LoadBalancer   172.20.134.203   internal-1a1bc15f63068a05e16dc10d31bfc8d4-628860359.eu-west-1.elb.amazonaws.com   80:31066/TCP   3d8h

Posted on 17/02/2021

Categories