Getting started with Argo Workflows
6 min read | by Jordi Prats
Argo Workflows is a open-source container-native workflow engine designed to run containerized jobs in Kubernetes clusters, similar to tekton.
It uses DAGs (Directed Acyclic Graphs) or step-based workflows that will run each in a container.
Installing Argo Workflows
First, we need to set the version we want to install, we can do it by setting the environment variable ARGO_WORKFLOWS_VERSION to the desired version:
export ARGO_WORKFLOWS_VERSION="v3.5.11"
Then we can install it to the argo namespace using the following commands:
kubectl create namespace argo
kubectl apply -n argo -f "https://github.com/argoproj/argo-workflows/releases/download/${ARGO_WORKFLOWS_VERSION}/quick-start-minimal.yaml"
We can now wait for the pods to be up and running, keep an eye on the argo-server pod:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
argo-server-76c65cd446-2lc82 0/1 Running 0 30s
httpbin-7b48b49985-fg7mb 1/1 Running 0 30s
minio-68dc5544c4-7lbsl 1/1 Running 0 30s
workflow-controller-68d7b854cc-6spmj 1/1 Running 0 30s
$ get pods
NAME READY STATUS RESTARTS AGE
argo-server-76c65cd446-2lc82 1/1 Running 0 96s
httpbin-7b48b49985-fg7mb 1/1 Running 0 96s
minio-68dc5544c4-7lbsl 1/1 Running 0 96s
workflow-controller-68d7b854cc-6spmj 1/1 Running 0 96s
Argo workflows GUI
We can access the Argo Workflows GUI by port-forwarding the argo-server service:
kubectl -n argo port-forward service/argo-server 2746:2746
We'll be able to access the GUI by visiting the localhost on port 2746. Make sure to explicitly set the protocol to https. So, navigate to https://localhost:2746 in your browser.
Hello World workflow
We are going to create a simple workflow that will run a container that will print hello world using the docker/whalesay image. This would be the yaml definition for the workflow:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: whalesay-
spec:
entrypoint: whalesay
templates:
- name: whalesay
container:
image: docker/whalesay:latest
command: [cowsay]
args: ["hello world"]
Bear in mind that since we want to be able to run the workflow multiple times, we need to set the generateName field in the metadata to a value that will be unique for each run. This also means that we'll need to use kubectl create instead of kubectl apply to create the workflow:
$ kubectl create -f hello-world.yaml ; kubectl get workflow -w
workflow.argoproj.io/whalesay-gsmmr created
NAME STATUS AGE MESSAGE
whalesay-gsmmr Running 1s
whalesay-gsmmr Succeeded 10s
As soon as it finishes, we can check the logs of the pod to see the output:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
argo-server-76c65cd446-2lc82 1/1 Running 0 49m
httpbin-7b48b49985-fg7mb 1/1 Running 0 49m
minio-68dc5544c4-7lbsl 1/1 Running 0 49m
whalesay-gsmmr 0/2 Completed 0 28s
workflow-controller-68d7b854cc-6spmj 1/1 Running 0 49m
$ kubectl logs whalesay-gsmmr
time="2024-10-21T04:04:24.975Z" level=info msg="capturing logs" argo=true
_____________
< hello world >
-------------
\
\
\
## .
## ## ## ==
## ## ## ## ===
/""""""""""""""""___/ ===
~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ / ===- ~~~
\______ o __/
\ \ __/
\____\______/
time="2024-10-21T04:04:25.976Z" level=info msg="sub-process exited" argo=true error="<nil>"
Multi-step workflow
We are now going to create a multi-step workflow that will generate a random number and create a configmap with that number.
To do so, first we'll need to assign additional permissions to the default service account in the argo namespace. We can do so by creating the following role and role binding that will grant the necessary permissions:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: argo
name: configmap-manager
rules:
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: configmap-manager-binding
namespace: argo
subjects:
- kind: ServiceAccount
name: default
namespace: argo
roleRef:
kind: Role
name: configmap-manager
apiGroup: rbac.authorization.k8s.io
We can apply the above yaml file to create both objects:
$ kubectl apply -f sa-cm.yaml
role.rbac.authorization.k8s.io/configmap-manager created
rolebinding.rbac.authorization.k8s.io/configmap-manager-binding created
Once we have this in place, we can create the workflow that will generate a random number and create a configmap with that number. Please notice how we are using the generate-and-create entrypoint that will run two steps: generate-random and create-configmap. The generate-random step will run a python script that will generate a random number between 1 and 100, while the create-configmap step will create a configmap with that number:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: random-configmap-
spec:
entrypoint: generate-and-create
templates:
- name: generate-and-create
steps:
- - name: generate-random-number
template: generate-random
- - name: create-configmap
template: create-configmap
arguments:
parameters:
- name: random-number
value: "{{steps.generate-random-number.outputs.result}}"
- name: generate-random
script:
image: python:3.9
command: [python]
source: |
import random
random_number = random.randint(1, 100)
print(random_number)
outputs:
result: "{{outputs.result}}"
- name: create-configmap
inputs:
parameters:
- name: random-number
container:
image: bitnami/kubectl:latest
command: [sh, -c]
args:
- |
kubectl create configmap random-configmap \
--from-literal=random-number={{inputs.parameters.random-number}} \
-n argo
We can now create the workflow using the following command:
$ kubectl create -f random-configmap.yaml ; kubectl get workflow -w
workflow.argoproj.io/random-configmap-kg8xk created
NAME STATUS AGE MESSAGE
whalesay-gsmmr Succeeded 20m
random-configmap-kg8xk Running 0s
random-configmap-kg8xk Running 10s
random-configmap-kg8xk Succeeded 20s
Once it finishes, we can check the configmap to see the random number that was generated:
$ kubectl get cm random-configmap
NAME DATA AGE
random-configmap 1 40s
$ kubectl get cm random-configmap -o yaml
apiVersion: v1
data:
random-number: "34"
kind: ConfigMap
metadata:
creationTimestamp: "2024-10-21T04:24:36Z"
name: random-configmap
namespace: argo
resourceVersion: "424750"
uid: 6e00ce29-b3b9-4c43-a3e2-33f12106d743
Checking the pods, we can see how it ran the two steps in different Pods:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
argo-server-76c65cd446-2lc82 1/1 Running 0 71m
httpbin-7b48b49985-fg7mb 1/1 Running 0 71m
minio-68dc5544c4-7lbsl 1/1 Running 0 71m
random-configmap-kg8xk-create-configmap-2480388875 0/2 Completed 0 2m39s
random-configmap-kg8xk-generate-random-1513880492 0/2 Completed 0 2m49s
whalesay-gsmmr 0/2 Completed 0 22m
workflow-controller-68d7b854cc-6spmj 1/1 Running 0 71m
In the status field of the workflow, we can see the inputs and outputs of the steps that were run:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
annotations:
workflows.argoproj.io/pod-name-format: v2
creationTimestamp: "2024-10-21T04:24:23Z"
generateName: random-configmap-
generation: 4
labels:
workflows.argoproj.io/completed: "true"
workflows.argoproj.io/phase: Succeeded
name: random-configmap-kg8xk
namespace: argo
resourceVersion: "424770"
uid: 7488c865-e03d-449a-bef0-d9d27669c529
spec:
(...)
status:
(...)
finishedAt: "2024-10-21T04:24:43Z"
nodes:
random-configmap-kg8xk:
(...)
random-configmap-kg8xk-1513880492:
(...)
outputs:
artifacts:
- name: main-logs
s3:
key: random-configmap-kg8xk/random-configmap-kg8xk-generate-random-1513880492/main.log
exitCode: "0"
result: "34"
(...)
random-configmap-kg8xk-2480388875:
(...)
inputs:
parameters:
- name: random-number
value: "34"
(...)
random-configmap-kg8xk-3320070688:
(...)
random-configmap-kg8xk-3387034069:
(...)
phase: Succeeded
progress: 2/2
resourcesDuration:
cpu: 0
memory: 4
startedAt: "2024-10-21T04:24:23Z"
taskResultsCompletionStatus:
random-configmap-kg8xk-1513880492: true
random-configmap-kg8xk-2480388875: true
Posted on 22/10/2024