Write constraints for OPA gatekeeper

Kubernetes OPA gatekeeper custom rule

3 min read | by Jordi Prats

Once we have OPA gatekeeper installed we might want to start writing our own rules if we cannot find it in the gatekeeper library.

To create a custom rule we'll have to create a ConstraintTemplate that will contain the relevant rego code and an instance of it to really start using it.

Browsing the gatekeeper library can be a great source of inspiration, even if the rules it contains doesn't do what you need you can still find code that can be adapted to do what you need:

The following code is making sure a Pod uses a set of tags:

        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_].key}
          missing := required - provided
          count(missing) > 0
          def_msg := sprintf("you must provide labels: %v", [missing])
          msg := get_message(input.parameters, def_msg)
        }

If what we need is to disallow some tolerations, we can change it's logic to fit our needs:

        violation[{"msg": msg, "details": {"restricted annotations": "found"}}] {
            provided := {toleration_in | toleration_in := input.review.object.spec.tolerations[_].key}
            excluded := {toleration_ex | toleration_ex := input.parameters.tolerations[_].key}
            diff := provided - excluded
            count(provided) != count(diff)
            msg := "found restricted toleration(s)"
        }

Having the code we just need to insert it on a ConstraintTemplate object with the proper openAPIV3Schema to hold the tolerations we want to restrict:

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: restricttolerations
spec:
  crd:
    spec:
      names:
        kind: RestrictTolerations
      validation:
        openAPIV3Schema:
          type: object
          properties:
            message:
              type: string
            tolerations:
              type: array
              description: A list of tolerations to deny
              items:
                type: object
                properties:
                  key:
                    type: string
                    description: Toleration key
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package restricttolerations

        violation[{"msg": msg, "details": {"restricted annotations": "found"}}] {
            provided := {toleration_in | toleration_in := input.review.object.spec.tolerations[_].key}
            excluded := {toleration_ex | toleration_ex := input.parameters.tolerations[_].key}
            diff := provided - excluded
            count(provided) != count(diff)
            msg := "found restricted toleration(s)"
        }

Once we have this object available we'll be able to create instances of it configuring it:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: RestrictTolerations
metadata:
  name: restrict-tolerations
  annotations:
    "helm.sh/hook": "post-install,post-upgrade"
    "helm.sh/hook-delete-policy": "before-hook-creation"
    "helm.sh/hook-weight": "1"
spec:
  enforcementAction: deny
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  parameters:
    tolerations:
      - key: node-role.kubernetes.io/demo
      - key: node-role.kubernetes.io/another
      - key: node-role.kubernetes.io/yetanother

We can now try creating a Pod with one of the disallowed tolerations

apiVersion: v1
kind: Pod
metadata:
  name: pod-tolerations-test
spec:
  containers:
  - name: nginx
    image: nginx
  tolerations:
  - key: "node-role.kubernetes.io/demo"
    operator: "Equal"
    value: "whatever"
    effect: "NoSchedule"

When trying to apply this Pod definition, we'll get an error from the gatekeeper's admission webhook disallowing the creation of this Pod:

$ kubectl apply -f testPod.yaml
Error from server (Forbidden): error when creating "testPod.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [restrict-tolerations] found restricted toleration(s)

Posted on 01/11/2022

Categories