OpenShift Pipelines - Tekton Introduction

OpenShift Tekton

- Thomas Jungbauer Thomas Jungbauer ( Lastmod: 2024-05-08 ) - 6 min read

OpenShift Pipelines is a cloud-native, continuous integration and delivery (CI/CD) solution for building pipelines using Tekton. Tekton is a flexible, Kubernetes-native, open-source CI/CD framework that enables automating deployments across multiple platforms (Kubernetes, serverless, VMs, etc) by abstracting away the underlying details. [1]

OpenShift Pipelines features

Source: [1]

  • Standard CI/CD pipeline definition based on Tekton

  • Build images with Kubernetes tools such as S2I, Buildah, Buildpacks, Kaniko, etc

  • Deploy applications to multiple platforms such as Kubernetes, serverless and VMs

  • Easy to extend and integrate with existing tools

  • Scale pipelines on-demand

  • Portable across any Kubernetes platform

  • Designed for microservices and decentralized teams

  • Integrated with the OpenShift Developer Console

Prerequisites

  1. OpenShift 4.x cluster. Try yourself at https://try.openshift.com

  2. Optional: Tekton CLI - Optional for now, since you could do everything via UI as well.

  3. oc/kubectl CLI or WebUI

The Tekton CLI is optional in case you prefer to do everything via the OpenShift WebUI. However, below examples make use of the Tekton CLI and I personally would recommend to at least install it. (Like oc client it is good to have a CLI option as well). In any case, all described action can be done directly via the WebUI as well.

Basic Concepts

Tekton makes use of several custom resources (CRD).

These CRDs are:

  1. Task: each step in a pipeline is a task, while a task can contain several steps itself, which are required to perform a specific task. For each Task a pod will be allocated and for each step inside this Task a container will be used. This helps in better scalability and better performance throughout the pipeline process.

  2. Pipeline: is a series of tasks, combined to work together in a defined (structured) way

  3. TaskRun: is the result of a Task, all combined TaskRuns are used in the PipelineRun

  4. PipelineRun: is the actual execution of a whole Pipeline, containing the results of the pipeline (success, failed…​)

Pipelines and Tasks should be generic and never define possible variables, like input git repository, directly in their definition. For this, the concept of PipelineResources has been created, which defines these parameters and which are used during a PipelineRun.

Installation

The OpenShift Pipeline is an operator which can e installed using the following yaml:

cat <<'EOF' > deploy-OpenShift-Pipelines.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: openshift-pipelines-operator
  namespace: openshift-operators
spec:
  channel: dev-preview
  name: openshift-pipelines-operator
  source: community-operators
  sourceNamespace: openshift-marketplace
EOF
oc create -f deploy-OpenShift-Pipelines.yaml

As an alternative, you can also use the WebUI to rollout the operator:

  1. Search for "OpenShift Pipeline" under OperatorHub and install it

  2. Select:

    1. All Namespaces: since the operator needs to watch for Tekton Custom Resources across all namespaces.

    2. Channel: Dev Preview

    3. Approve Strategy: Automatic

Prepare a tutorial project

To test our OpenShift Pipelines, we need to deploy an example application. This application let’s you vote what pet you like more: Cats or Dogs? It contais of a backend and a frontend part, which both will be deployed in a namespace.

Let’s first create a new project:

oc new-project pipelines-tutorial

The OpenShift Pipeline operator will automatically create a pipeline serviceaccount with all required permissions to build and push an image and which is used by PipelineRuns:

oc get sa pipeline
NAME       SECRETS   AGE
pipeline   2         15s

Create a Task

A Task is the smallest block of a Pipeline which by itself can contain one or more steps which are executed in order to process a specific element. For each Task a pod is allocated and each step is running in a container inside this pod. Tasks are reusable by other Pipelines. Input and Output specifications can be used to interact with other Tasks.

Let’s create two tasks Source: Pipeline-Tutorial

cat <<'EOF' > deploy-Example-Tasks.yaml
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: apply-manifests
spec:
  inputs:
    resources:
      - {type: git, name: source}
    params:
      - name: manifest_dir
        description: The directory in source that contains yaml manifests
        type: string
        default: "k8s"
  steps:
    - name: apply
      image: quay.io/openshift/origin-cli:latest
      workingDir: /workspace/source
      command: ["/bin/bash", "-c"]
      args:
        - |-
          echo Applying manifests in $(inputs.params.manifest_dir) directory
          oc apply -f $(inputs.params.manifest_dir)
          echo -----------------------------------
---
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: update-deployment
spec:
  inputs:
    resources:
      - {type: image, name: image}
    params:
      - name: deployment
        description: The name of the deployment patch the image
        type: string
  steps:
    - name: patch
      image: quay.io/openshift/origin-cli:latest
      command: ["/bin/bash", "-c"]
      args:
        - |-
          oc patch deployment $(inputs.params.deployment) --patch='{"spec":{"template":{"spec":{
            "containers":[{
              "name": "$(inputs.params.deployment)",
              "image":"$(inputs.resources.image.url)"
            }]
          }}}}'
EOF
oc create -f deploy-Example-Tasks.yaml

Verify that the two tasks have been created using the Tekton CLI:

tkn task ls

NAME                AGE
apply-manifests     52 seconds ago
update-deployment   52 seconds ago

Create a Pipeline

A pipeline is a set of Tasks, which should be executed in a defined way to achieve a specific goal.

The example Pipeline below uses two resources:

  • git-repo: defines the Git-Source

  • image: Defines the target at a repository

It first uses the Task buildah, which is a standard Task the OpenShift operator created automatically. This task will build the image. The resulted image is pushed to an image registry, defined in the output parameter. After that our created tasks apply-manifest and update-deployment are executed. The execution order of these tasks is defined with the runAfter Parameter in the yaml definition.

The Pipeline should be re-usable accross multiple projects or environments, thats why the resources (git-repo and image) are not defined here. When a Pipeline is executed, these resources will get defined.
cat <<'EOF' > deploy-Example-Pipeline.yaml
apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: build-and-deploy
spec:
  resources:
  - name: git-repo
    type: git
  - name: image
    type: image
  params:
  - name: deployment-name
    type: string
    description: name of the deployment to be patched
  tasks:
  - name: build-image
    taskRef:
      name: buildah
      kind: ClusterTask
    resources:
      inputs:
      - name: source
        resource: git-repo
      outputs:
      - name: image
        resource: image
    params:
    - name: TLSVERIFY
      value: "false"
  - name: apply-manifests
    taskRef:
      name: apply-manifests
    resources:
      inputs:
      - name: source
        resource: git-repo
    runAfter:
    - build-image
  - name: update-deployment
    taskRef:
      name: update-deployment
    resources:
      inputs:
      - name: image
        resource: image
    params:
    - name: deployment
      value: $(params.deployment-name)
    runAfter:
    - apply-manifests
EOF
oc create -f deploy-Example-Pipeline.yaml

Verify that the Pipeline has been created using the Tekton CLI:

tkn pipeline ls

NAME               AGE             LAST RUN   STARTED   DURATION   STATUS
build-and-deploy   3 seconds ago   ---        ---       ---        ---

Trigger Pipeline

After the Pipeline has been created, it can be triggered to execute the Tasks.

Create PipelineResources

Since the Pipeline is generic, we need to define 2 PipelineResources first, to execute a Pipepline. Our example application contains a frontend (vote-ui) AND a backend (vote-api), therefore 4 PipelineResources will be created. (2 times git repository to clone the source and 2 time output image)

Quick overview:

  • ui-repo: will be used as git_repo in the Pipepline for the Frontend

  • ui-image: will be used as image in the Pipeline for the Frontend

  • api-repo: will be used as git_repo in the Pipepline for the Backend

  • api-image: will be used as image in the Pipeline for the Backend

cat <<'EOF' > deploy-Example-PipelineResources.yaml
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: ui-repo
spec:
  type: git
  params:
  - name: url
    value: http://github.com/openshift-pipelines/vote-ui.git
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: ui-image
spec:
  type: image
  params:
  - name: url
    value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-ui:latest
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: api-repo
spec:
  type: git
  params:
  - name: url
    value: http://github.com/openshift-pipelines/vote-api.git
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: api-image
spec:
  type: image
  params:
  - name: url
    value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-api:latest
EOF
oc create -f deploy-Example-PipelineResources.yaml

The resources can be listed with:

tkn resource ls

NAME        TYPE    DETAILS
api-repo    git     url: http://github.com/openshift-pipelines/vote-api.git
ui-repo     git     url: http://github.com/openshift-pipelines/vote-ui.git
api-image   image   url: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-api:latest
ui-image    image   url: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-ui:latest

Execute Pipelines

We start a PipelineRune for the backend and frontend of our application.

cat <<'EOF' > deploy-Example-PipelineRun.yaml
apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
  name: build-deploy-api-pipelinerun
spec:
  pipelineRef:
    name: build-and-deploy
  resources:
  - name: git-repo
    resourceRef:
      name: api-repo
  - name: image
    resourceRef:
      name: api-image
  params:
  - name: deployment-name
    value: vote-api
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
  name: build-deploy-ui-pipelinerun
spec:
  pipelineRef:
    name: build-and-deploy
  resources:
  - name: git-repo
    resourceRef:
      name: ui-repo
  - name: image
    resourceRef:
      name: ui-image
  params:
  - name: deployment-name
    value: vote-ui
EOF
oc create -f deploy-Example-PipelineRun.yaml

The PipelineRuns can be listed with

tkn pipelinerun ls

NAME                           STARTED         DURATION   STATUS
build-deploy-api-pipelinerun   3 minutes ago   ---        Running
build-deploy-ui-pipelinerun    3 minutes ago   ---        Running

Moreover, the logs can be viewed with the following command and select the appropriate PipelineRun:

tkn pipeline logs -f
? Select pipelinerun:  [Use arrows to move, type to filter]
> build-deploy-api-pipelinerun started 2 minutes ago
  build-deploy-ui-pipelinerun started 2 minutes ago

Checking your application

Now our Pipeline built and deployed the voting application, where you can vote if you prefere cats or dogs (Cats or course :) )

Get the route of your project and open the URL in the browser. (Should be something like vote-ui-pipelines-tutorial.apps.yourclustername)

Tekton Vote App
Figure 1. Tekton: Example Application

OpenShift WebUI

With the OpenShift Pipeline operator a new menu item is introduced on the WebUI of OpenShift. All Tekton CLI command which are used above, can actually be replaced with the web interface, in case you prefere this. The big advantage is th graphical presentation of Pipelines and their lifetime.

I will not create screenshots for every screen, but for example pipelines:

Under Pipelines a list of pipelines will be shown.

Pipelines
Figure 2. OpenShift UI: List of Pipelines

Additional Resources

Sources

See Also:

  • Using Kustomize to post render a Helm Chart –

    Lately I came across several issues where a given Helm Chart must be modified after it has been rendered by Argo CD. Argo CD does a helm template to render a Chart. Sometimes, especially when you work with Subcharts or when a specific setting is not yet supported by the Chart, you need to modify it later …​ you need to post-render the Chart. In this very short article, I would like to demonstrate this on a real-live example I had to do. I would like to inject annotations to a Route objects, so that the certificate can be injected. This is done by the cert-utils operator. For the post-rendering the Argo CD repo pod will be extended with a sidecar container, that is watching for the repos and patches them if required.

  • Managing Certificates using GitOps approach –

    The article SSL Certificate Management for OpenShift on AWS explains how to use the Cert-Manager Operator to request and install a new SSL Certificate. This time, I would like to leverage the GitOps approach using the Helm Chart cert-manager I have prepared to deploy the Operator and order new Certificates. I will use an ACME Letsencrypt issuer with a DNS challenge. My domain is hosted at AWS Route 53. However, any other integration can be easily used.

  • Update Cluster Version using GitOps approach –

    During a GitOps journey at one point, the question arises, how to update a cluster? Nowadays it is very easy to update a cluster using CLI or WebUI, so why bother with GitOps in that case? The reason is simple: Using GitOps you can be sure that all clusters are updated to the correct, required version and the version of each cluster is also managed in Git. All you need is the channel you want to use and the desired cluster version. Optionally, you can define the exact image SHA. This might be required when you are operating in a restricted environment.

  • Multiple Sources for Applications in Argo CD –

    Argo CD or OpenShift GitOps uses Applications or ApplicationSets to define the relationship between a source (Git) and a cluster. Typically, this is a 1:1 link, which means one Application is using one source to compare the cluster status. This can be a limitation. For example, if you are working with Helm Charts and a Helm repository, you do not want to re-build (or re-release) the whole chart just because you made a small change in the values file that is packaged into the repository. You want to separate the configuration of the chart with the Helm package. The most common scenarios for multiple sources are (see: Argo CD documentation): Your organization wants to use an external/public Helm chart You want to override the Helm values with your own local values You don’t want to clone the Helm chart locally as well because that would lead to duplication and you would need to monitor it manually for upstream changes. This small article describes three different ways with a working example and tries to cover the advantages and disadvantages of each of them. They might be opinionated but some of them proved to be easier to use and manage.

  • Installing OpenShift Logging using GitOps –

    OpenShift Logging is one of the more complex things to install and configure on an OpenShift cluster. Not because the service or Operators are so complex to understand, but because of the dependencies logging has. Besides the logging operator itself, the Loki operator is required, the Loki operator requires access to an object storage, that might be configured or is already available. In this article, I would like to demonstrate the configuration of the full stack using an object storage from OpenShift Data Foundation. This means: Installing the logging operator into the namespace openshift-logging Installing the Loki operator into the namespace openshift-operators-redhat Creating a new BackingStore and BucketClass Generating the Secret for Loki to authenticate against the object storage Configuring the LokiStack resource Configuring the ClusterLogging resource All steps will be done automatically. In case you have S3 storage available, or you are not using OpenShift Data Foundation, the setup will be a bit different. For example, you do not need to create a BackingStore or the Loki authentication Secret.

Copyright © 2020 - 2024 Toni Schmidbauer & Thomas Jungbauer