Building a Kubernetes operator using operator-sdk (golang)

Kubernetes operator-sdk

7 min read | by Jordi Prats

With the operator pattern we are going to define the state we want it to be an let the controller do changes on it so the current state matches the desired state

Using the operator-sdk we can focus on the code that makes this happen rather than on all the boiler-plate that is going to be involved on it

To start a operator-sdk project we are going to need some prerequisites:


We can check the operator-sdk documentation on it's installation for go, but in a nutshell we'll need:

  • Go version 1.18 or later:
  • A test cluster, such as minikube

Install operator-sdk

To be able to create all the boiler-plate we'll need to install the operator-sdk cli:

export ARCH=$(case $(uname -m) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(uname -m) ;; esac)
export OS=$(uname | awk '{print tolower($0)}')
curl -LO ${OPERATOR_SDK_DL_URL}/operator-sdk_${OS}_${ARCH}
chmod +x operator-sdk_${OS}_${ARCH} && sudo mv operator-sdk_${OS}_${ARCH} /usr/local/bin/operator-sdk

Init project

We are going to use the pet2cattle/operatorsdk-example repository as an example:

$ git clone
Cloning into 'operatorsdk-example'...
warning: You appear to have cloned an empty repository.
$ cd operatorsdk-example

Then we'll be able to initialize it using operator-sdk init:

$ operator-sdk init --domain --repo
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
Get controller runtime:
$ go get
go: downloading v0.0.0-20220210224613-90d013bbcef8
go: downloading v0.0.0-20220127200216-cd36cc0744dd
go: downloading v0.0.0-20211104180415-d3ed0bb246c8
Update dependencies:
$ go mod tidy
go: downloading v0.11.18
go: downloading v0.9.13
go: downloading v0.81.0
go: downloading v1.1.0
go: downloading v14.2.0+incompatible
go: downloading v0.0.0-20220214200702-86341886e292
go: downloading v0.2.1
go: downloading v0.4.1
go: downloading v0.6.0
go: downloading v3.2.3+incompatible
go: downloading v0.3.0
Next: define a resource with:
$ operator-sdk create api

We can check commit 2822dd5 to check all the files that it creates

Create API

Then, we can use operator-sdk create api to create the CRD resources that we are going to use to define the desired state of out application:

$ operator-sdk create api --group example --version v1 --kind Demo --resource --controller
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
Update dependencies:
$ go mod tidy
go: downloading v2.0.0
Running make:
$ make generate
mkdir -p /home/pet2cattle/git/operatorsdk-example/bin
GOBIN=/home/pet2cattle/git/operatorsdk-example/bin go install
go: downloading v0.9.0
go: downloading v1.4.0
go: downloading v0.1.10-0.20220218145154-897bd77cd717
go: downloading v1.12.0
go: downloading v0.2.5
go: downloading v0.0.12
go: downloading v0.1.8
go: downloading v0.6.0-dev.0.20220106191415-9b9b3d81d5e3
/home/pet2cattle/git/operatorsdk-example/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifests

We can also check commit b7eeb09 to see what this is going under the hood.

Define schema

At this point we'll need to start modifying the files that the operator-sdk cli have created.

First, we'll need to update the api/v1/demo_types.go file with whatever we want to to contain. For this example we are going to use the attribute name as an example:

// DemoSpec defines the desired state of Demo
type DemoSpec struct {
  Name string `json:"name"`

Then, we can update the sample instance using this file: config/samples/example_v1_expiration.yaml:

kind: Demo
  name: demo-sample
  example: 'this-is-an-example'

See the change on github: commit 0cc82f7

Installing the CRD

Using make install we can install the CRD to the active cluster (we are using minikube for this example)

$ make install
/home/pet2cattle/git/operatorsdk-example/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
curl -s "" | bash -s -- 3.8.7 /home/pet2cattle/git/operatorsdk-example/bin
{Version:kustomize/v3.8.7 GitCommit:ad092cc7a91c07fdf63a2e4b7f13fa588a39af4f BuildDate:2020-11-11T23:14:14Z GoOs:linux GoArch:amd64}
kustomize installed to /home/pet2cattle/git/operatorsdk-example/bin/kustomize
/home/pet2cattle/git/operatorsdk-example/bin/kustomize build config/crd | kubectl apply -f - created

Deploying a CRD instance

We can use kustomize to apply it for more complex projects:

$ kustomize build config/samples | kubectl apply -f - created

But for now, kubectl apply would work just as well:

$ kubectl apply -f config/samples/example_v1_expiration.yaml created

Spin up controller

Using make run we can spin up the controller, but at this stage it's just a placeholder: It's not going to do much:

$ make run
GOBIN=/home/pet2cattle/git/operatorsdk-example/bin go install
/home/pet2cattle/git/operatorsdk-example/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/pet2cattle/git/operatorsdk-example/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go run ./main.go
1.6556603319415438e+09  INFO  controller-runtime.metrics  Metrics server is starting to listen  {"addr": ":8080"}
1.6556603319434564e+09  INFO  setup starting manager
1.6556603319438052e+09  INFO  Starting server {"kind": "health probe", "addr": "[::]:8081"}
1.655660331943896e+09 INFO  Starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}
1.6556603319438577e+09  INFO  Starting EventSource  {"controller": "demo", "controllerGroup": "", "controllerKind": "Demo", "source": "kind source: *v1.Demo"}
1.6556603319439232e+09  INFO  Starting Controller {"controller": "demo", "controllerGroup": "", "controllerKind": "Demo"}
1.6556603320442946e+09  INFO  Starting workers  {"controller": "demo", "controllerGroup": "", "controllerKind": "Demo", "worker count": 1}

Check resource

We can check the CRD using the usual kubectl get demo

$ kubectl get demo
NAME          AGE
demo-sample   88s

Or even describe it:

$ kubectl describe demo demo-sample
Name:         demo-sample
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:
Kind:         Demo
  Creation Timestamp:  2022-06-19T19:37:51Z
  Generation:          1
  Managed Fields:
    API Version:
    Fields Type:  FieldsV1

    Manager:         kubectl-client-side-apply
    Operation:       Update
    Time:            2022-06-19T17:37:51Z
  Resource Version:  552
  UID:               67d2f0ed-d1d0-46a5-930a-70186717cfc6
  Example:  this-is-an-example
Events:     <none>

Implement reconcile loop

If we want to actually implement something we'll have to update the Reconcile function on the controllers/expiration_controller.go file.

For this example we are just going to print some data to stdout:

func (r *ExpirationReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
  _ = log.FromContext(ctx)

  // Fetch the Expiration instance.
  expiration := &bestbyv1.Expiration{}
  if err := r.Get(ctx, req.NamespacedName, expiration); err != nil {
    return ctrl.Result{}, client.IgnoreNotFound(err)

  fmt.Printf("%#v\n", expiration)

  fmt.Printf("%#v\n", expiration.Spec.Lifetime)

  return ctrl.Result{}, nil

For this to work we'll have to import fmt, on the commit 0fbb9b7 you can see all the details.

By running the controller again with make run, we will be able to see how it is printing this information to stdout:

$ make run
/home/pet2cattle/git/operatorsdk-example/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/pet2cattle/git/operatorsdk-example/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go run ./main.go
1.6556605118827746e+09  INFO  controller-runtime.metrics  Metrics server is starting to listen  {"addr": ":8080"}
1.6556605118832474e+09  INFO  setup starting manager
1.655660511883873e+09 INFO  Starting server {"kind": "health probe", "addr": "[::]:8081"}
1.6556605118856435e+09  INFO  Starting EventSource  {"controller": "demo", "controllerGroup": "", "controllerKind": "Demo", "source": "kind source: *v1.Demo"}
1.6556605118860493e+09  INFO  Starting Controller {"controller": "demo", "controllerGroup": "", "controllerKind": "Demo"}
1.6556605118906796e+09  INFO  Starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}
1.6556605119876258e+09  INFO  Starting workers  {"controller": "demo", "controllerGroup": "", "controllerKind": "Demo", "worker count": 1}
&v1.Demo{TypeMeta:v1.TypeMeta{Kind:"Demo", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"demo-sample", GenerateName:"", Namespace:"default", SelfLink:"", UID:"67d2f0ed-d1d0-46a5-930a-70186717cfc6", ResourceVersion:"552", Generation:1, CreationTimestamp:time.Date(2022, time.June, 19, 19, 37, 51, 0, time.Local), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"":"{\"apiVersion\":\"\",\"kind\":\"Demo\",\"metadata\":{\"annotations\":{},\"name\":\"demo-sample\",\"namespace\":\"default\"},\"spec\":{\"example\":\"this-is-an-example\"}}\n"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubectl-client-side-apply", Operation:"Update", APIVersion:"", Time:time.Date(2022, time.June, 19, 19, 37, 51, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00043a348), Subresource:""}}}, Spec:v1.DemoSpec{Example:"this-is-an-example"}, Status:v1.DemoStatus{}}

Posted on 21/06/2022