4 min read | by Jordi Prats
On a kubernetes cluster not all nodes are expected to have the same amount of resources so we might need to schedule some pods on specific nodes due to the resources they have (for example access to a GPU) or due to the network connectivity they have (for example edge nodes). Using nodeSelector we can the scheduler how we want out pods to be scheduled
On another pods we saw how to assign a pod to a specific node but, generally speaking, we won't want to select a specific node: what we would like to do is to set a group of nodes on which we would like the pod to be scheduled.
To simulate this environment we are going to using a kubernetes cluster with multiple nodes using minkube:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready control-plane,master 8m v1.20.0
minikube-m02 Ready <none> 5m52s v1.20.0
minikube-m03 Ready <none> 4m23s v1.20.0
minikube-m04 Ready <none> 3m9s v1.20.0
minikube-m04 Ready <none> 113s v1.20.0
To classify the nodes we are going to use labels. We can set them using kubectl label as follows:
kubectl label nodes <nodename> <label>=<value>
For example, for setting a label called "nodegroup" setting it's value to "alpha" for node minikube-m04 and minikube-m05 we would execute the following commands:
kubectl label nodes minikube-m04 nodegroup=alpha
kubectl label nodes minikube-m05 nodegroup=alpha
To be able to use these labels to schedule a pod we will have to use a nodeSelector key on the spec section. Under this key we will have to add all the labels we want to use to schedule this pod (we can also set it on a deployment on the spec.template.spec section). For example:
apiVersion: v1
kind: Pod
metadata:
name: demo
spec:
nodeSelector:
nodegroup: alpha
containers:
- name: demo
image: busybox
command: ["sleep"]
args: ["1h"]
By deploying this manifest the scheduler will search for a node that matches the nodeSelector labels:
$ kubectl apply -f demo.yaml
pod/demo created
Using the -o wide option we will be able to see the selected node:
$ kubectl get pod demo -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
demo 1/1 Running 0 19s 10.244.3.2 minikube-m04 <none> <none>
We can also use this manifest to create multiple pods to see where it is scheduled:
$ for i in $(seq 1 10); do cat demo.yaml | sed "s/demo/demo-$i/g" | kubectl apply -f -; done
pod/demo-1 created
pod/demo-2 created
pod/demo-3 created
(...)
We will be able to see that only the pods with the label (minikube-m04 and minikube-m05) are going to be selected regardless of any empty node on the cluster
$ kubectl get pod -o wide | grep demo
demo 1/1 Running 0 2m10s 10.244.3.2 minikube-m04 <none> <none>
demo-1 1/1 Running 0 33s 10.244.4.2 minikube-m05 <none> <none>
demo-10 0/1 ContainerCreating 0 23s <none> minikube-m05 <none> <none>
demo-2 1/1 Running 0 32s 10.244.3.3 minikube-m04 <none> <none>
demo-3 1/1 Running 0 32s 10.244.4.3 minikube-m05 <none> <none>
demo-4 1/1 Running 0 30s 10.244.3.4 minikube-m04 <none> <none>
demo-5 1/1 Running 0 29s 10.244.4.4 minikube-m05 <none> <none>
demo-6 0/1 ContainerCreating 0 27s <none> minikube-m05 <none> <none>
demo-7 0/1 ContainerCreating 0 27s <none> minikube-m05 <none> <none>
demo-8 1/1 Running 0 26s 10.244.3.5 minikube-m04 <none> <none>
demo-9 1/1 Running 0 24s 10.244.3.6 minikube-m04 <none> <none>
Posted on 18/03/2021