3 min read | by Jordi Prats
On a kubernetes cluster you might find the following error:
$ kubectl apply -f ingress.yaml
Error from server (InternalError): error when creating "ingress": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post https://ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1beta1/ingresses?timeout=10s: service "ingress-nginx-controller-admission" not found
This message is telling you that the nginx ingress controller is not responding: This can be due to the fact that it might have partially uninstalled it (for example, by removing the namespace) but some of the global objects still present (for example the ValidatingWebhookConfiguration object).
To remove the ValidatingWebhookConfiguration object for the nginx ingress controller is named ingress-nginx-admission, we can remove it with kubectl delete like this:
$ kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission
validatingwebhookconfiguration.admissionregistration.k8s.io "ingress-nginx-admission" deleted
Once you have it removed, you will be able to create ingress objects as usual:
$ kubectl apply -f ingress.yaml
ingress.extensions/testingress created
We should be considering this a workaround, we need to keep in mind that removing a namespace is not equivalent of uninstalling an application: Some applications might have created objects that are not namespaced:
$ kubectl api-resources --namespaced=false
NAME SHORTNAMES APIVERSION NAMESPACED KIND
componentstatuses cs v1 false ComponentStatus
namespaces ns v1 false Namespace
nodes no v1 false Node
persistentvolumes pv v1 false PersistentVolume
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition
apiservices apiregistration.k8s.io/v1 false APIService
tokenreviews authentication.k8s.io/v1 false TokenReview
selfsubjectaccessreviews authorization.k8s.io/v1 false SelfSubjectAccessReview
selfsubjectrulesreviews authorization.k8s.io/v1 false SelfSubjectRulesReview
subjectaccessreviews authorization.k8s.io/v1 false SubjectAccessReview
certificatesigningrequests csr certificates.k8s.io/v1 false CertificateSigningRequest
eniconfigs crd.k8s.amazonaws.com/v1alpha1 false ENIConfig
ingressclassparams elbv2.k8s.aws/v1beta1 false IngressClassParams
flowschemas flowcontrol.apiserver.k8s.io/v1beta1 false FlowSchema
prioritylevelconfigurations flowcontrol.apiserver.k8s.io/v1beta1 false PriorityLevelConfiguration
nodes metrics.k8s.io/v1beta1 false NodeMetrics
ingressclasses networking.k8s.io/v1 false IngressClass
runtimeclasses node.k8s.io/v1 false RuntimeClass
podsecuritypolicies psp policy/v1beta1 false PodSecurityPolicy
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io/v1 false ClusterRole
priorityclasses pc scheduling.k8s.io/v1 false PriorityClass
csidrivers storage.k8s.io/v1 false CSIDriver
csinodes storage.k8s.io/v1 false CSINode
storageclasses sc storage.k8s.io/v1 false StorageClass
volumeattachments storage.k8s.io/v1 false VolumeAttachment
Thus, be aware that by removing the nginx ingress controller namespace and the ingress-nginx-admission webhook you are not completely removed all the objects that were installed with the nginx ingress controller: You also want to check for the presence of some clusterroles and clusterrolebindings to also get rid of them:
$ kubectl get clusterrole | grep nginx
ingress-nginx 2021-01-26T19:23:38Z
ingress-nginx-admission 2021-01-26T19:23:47Z
$ kubectl get clusterrolebinding | grep nginx
ingress-nginx ClusterRole/ingress-nginx 7d20h
ingress-nginx-admission ClusterRole/ingress-nginx-admission 7d20h
Posted on 26/02/2021