Troubleshoot Kubernetes service not being resolved

2 min read | by Jordi Prats

When you create a Kubernetes Service, pods from within the same namespace should be able to resolve it's IP by name. For example, if we create a service named ampa-votacions; any pod from the same namespace should be able to resolve it's IP. But sometimes it can't be resolved:

$ kubectl exec -it ampa-install-ws7cw -- sh
/ $ nslookup ampa-votacions
Server:   172.20.0.10
Address:  172.20.0.10:53

** server can't find ampa-votacions.us-west-2.compute.internal: NXDOMAIN

** server can't find ampa-votacions.ampa.svc.cluster.local: NXDOMAIN

** server can't find ampa-votacions.svc.cluster.local: NXDOMAIN

** server can't find ampa-votacions.ampa.svc.cluster.local: NXDOMAIN

** server can't find ampa-votacions.cluster.local: NXDOMAIN

** server can't find ampa-votacions.svc.cluster.local: NXDOMAIN

** server can't find ampa-votacions.cluster.local: NXDOMAIN

** server can't find ampa-votacions.us-west-2.compute.internal: NXDOMAIN

One common issue is that there are no endpoints available, we can check it using kubectl describe:

$ kubectl describe svc ampa-votacions
Name:              ampa-votacions
Namespace:         ampa
Labels:            app.kubernetes.io/instance=ampa
                   app.kubernetes.io/managed-by=Helm
                   app.kubernetes.io/name=ampa
                   app.kubernetes.io/version=1.19.4
                   component=votacions
                   helm.sh/chart=ampa-0.1.0
Annotations:       meta.helm.sh/release-name: ampa
                   meta.helm.sh/release-namespace: ampa
Selector:          app=ampa,component=votacions
Type:              ClusterIP
IP Families:       <none>
IP:                None
IPs:               <none>
Port:              votacions  8080/TCP
TargetPort:        8080/TCP
Endpoints:         <none>
Session Affinity:  None
Events:            <none>

We'll need to check:

  • What's going on with the pods / deployments what are supposed to be behind this service
  • Check that the pods / deployments are exposing the service
  • Check that the selectorLabels are matching the correct pods

Once we find and resolve the issue, we can check again the service using kubectl describe svc. If the issues is really fixed, we will be able to see the presence of endpoints:

$ kubectl describe svc ampa-votacions
Name:              ampa-votacions
Namespace:         ampa
Labels:            app.kubernetes.io/instance=ampa
                   app.kubernetes.io/managed-by=Helm
                   app.kubernetes.io/name=ampa
                   app.kubernetes.io/version=1.19.14
                   component=votacions
                   helm.sh/chart=ampa-0.1.0
Annotations:       meta.helm.sh/release-name: ampa
                   meta.helm.sh/release-namespace: ampa
Selector:          app=ampa,component=votacions
Type:              ClusterIP
IP Families:       <none>
IP:                None
IPs:               <none>
Port:              votacions  8080/TCP
TargetPort:        8080/TCP
Endpoints:         10.103.196.193:8080
Session Affinity:  None
Events:            <none>

Once we have endpoints available, we will be able to start resolving the service from the pods

Posted on 19/04/2021

Categories