Build container images (and push them to a registry) in Kubernetes with Tekton

tekton build push Kubernetes

5 min read | by Jordi Prats

We can build containers from within a Kubernetes cluster using Kaniko using Pods or use some framework to streamline the process such as Shipwright that uses tekton pipelines to actually run the process. We can skip Shipwright and create directly tekton pipelines.

Install tekton

First we'll need to install tekton, to do so we just need to apply the following manifests:

kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

Create secrets

To be able to work with private repositories and registries we'll have to push som secrets:

Private registry

To configure a private registry we'll need to create a secret with the following type: kubernetes.io/dockerconfigjson. The data it requires is the following:

kubectl create secret docker-registry private-registry \
                        --docker-server=$REGISTRY_SERVER \
                        --docker-username=$REGISTRY_USER \
                        --docker-password=$REGISTRY_PASS \
                        --docker-email=$REGISTRY_EMAIL

Private repository

In order to tell tekton to use a given private ssh key for github.com we'll need to create a secret as follows:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: github-key
  annotations:
    tekton.dev/git-0: github.com
type: kubernetes.io/ssh-auth
data:
  ssh-privatekey: $(base64 -w0 ~/.ssh/id_rsa)
EOF

Tekton ServiceAccount

In order to let tekton have these secrets we are going to create a ServiceAccount referencing the secrets:

cat <<"EOF" | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: pipeline-runner
secrets:
- name: private-registry
- name: github-key
EOF

Import tasks

To actually define what we want to run we can do so using tekton tasks either by defining it from scratch or reusing one from tekton hub. For this demo we are going to reuse publicly available git-clone and the kaniko tasks:

kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/git-clone/0.3/git-clone.yaml
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/kaniko/0.2/kaniko.yaml

We can check it's yaml definition to see what it basically does is to run a container. For example, the kaniko task looks like this:

$ kubectl get task kaniko -o yaml
apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: kaniko
(...)
spec:
  description: |-
    This Task builds source into a container image using Google's kaniko tool.
    Kaniko doesn't depend on a Docker daemon and executes each command within a Dockerfile completely in userspace. This enables building container images in environments that can't easily or securely run a Docker daemon, such as a standard Kubernetes cluster.
  params:
  - description: Name (reference) of the image to build.
    name: IMAGE
    type: string
  - default: ./Dockerfile
    description: Path to the Dockerfile to build.
    name: DOCKERFILE
    type: string
  - default: ./
    description: The build context used by Kaniko.
    name: CONTEXT
    type: string
  - default: ""
    name: EXTRA_ARGS
    type: string
  - default: gcr.io/kaniko-project/executor:v1.5.1@sha256:c6166717f7fe0b7da44908c986137ecfeab21f31ec3992f6e128fff8a94be8a5
    description: The image on which builds will run (default is v1.5.1)
    name: BUILDER_IMAGE
    type: string
  results:
  - description: Digest of the image just built.
    name: IMAGE-DIGEST
    type: string
  steps:
  - args:
    - $(params.EXTRA_ARGS)
    - --dockerfile=$(params.DOCKERFILE)
    - --context=$(workspaces.source.path)/$(params.CONTEXT)
    - --destination=$(params.IMAGE)
    - --oci-layout-path=$(workspaces.source.path)/$(params.CONTEXT)/image-digest
    computeResources: {}
    env:
    - name: DOCKER_CONFIG
      value: /tekton/home/.docker
    image: $(params.BUILDER_IMAGE)
    name: build-and-push
    securityContext:
      runAsUser: 0
    workingDir: $(workspaces.source.path)
(...)

Once we have them in the namespace, we'll be able to start creating the pipeline definition:

$ kubectl get tasks
NAME        AGE
git-clone   6s
kaniko      3s

Pipeline definition

The pipeline definition (Pipeline object) will hold that tasks that it is going to run and the parameters it is going to need to configure it:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: build-and-deploy-pipeline
spec:
  workspaces:
    - name: git-source
      description: The git repo
  params:
    - name: gitUrl
      description: Git repository url
      type: string
    - name: gitRevision
      description: Git revision to check out
      type: string
      default: master
    - name: pathToContext
      description: The path to the build context
      default: src
      type: string
    - name: imageUrl
      description: Image name including repository
      type: string
    - name: imageTag
      description: Image tag
      type: string
      default: latest
  tasks:
    - name: clone-repo
      taskRef:
        name: git-clone
      workspaces:
        - name: output
          workspace: git-source
      params:
        - name: url
          value: "$(params.gitUrl)"
        - name: revision
          value: "$(params.gitRevision)"
        - name: subdirectory
          value: "."
        - name: deleteExisting
          value: "true"
        - name: verbose
          value: "true"
    - name: build-and-push-image
      taskRef:
        name: kaniko
      runAfter:
        - clone-repo
      workspaces:
        - name: source
          workspace: git-source
      params:
        - name: CONTEXT
          value: $(params.pathToContext)
        - name: IMAGE
          value: $(params.imageUrl):$(params.imageTag)

Pipeline execution

To actually run the pipeline, building and pushing the container image to the private registry we'll have to create a PipelineRun object, specifying the ServiceAccount with the appropriate secrets:

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: build-and-deploy-execution
spec:
  pipelineRef:
    name: build-and-deploy-pipeline
  params:
    - name: gitUrl
      value: git@github.com:pet2cattle/demo-app.git
    - name: gitRevision
      value: HEAD
    - name: imageUrl
      value: repo.pet2cattle.com/demo-app
    - name: imageTag
      value: latest
    - name: pathToContext
      value: '.'
  serviceAccountName: pipeline-runner
  workspaces:
    - name: git-source
      volumeClaimTemplate:
        spec:
          accessModes: 
          - ReadWriteOnce
          resources:
            requests:
              storage: 1Gi

As soon as this object gets picked up, it will create a volumeClaim to hold the temporal data it uses and start running the Pods with each of the tasks that we have configured with the Pipeline template. We can check it's status using kubectl get pipelinerun:

$ kubectl get pipelinerun
NAME                         SUCCEEDED   REASON               STARTTIME   COMPLETIONTIME
build-and-deploy-execution   Unknown     Running              10s
$ kubectl get pipelinerun
NAME                         SUCCEEDED   REASON               STARTTIME   COMPLETIONTIME
build-and-deploy-execution   True        Succeeded            48m         44m

Posted on 04/01/2023

Categories