OpenShift Pipelines - Tekton Introduction

Thu, 16 Apr 2020 00:00:00 +0000

OpenShift Pipelines is a cloud-native, continuous integration and delivery (CI/CD) solution for building pipelines using Tekton. Tekton is a flexible, Kubernetes-native, open-source CI/CD framework that enables automating deployments across multiple platforms (Kubernetes, serverless, VMs, etc) by abstracting away the underlying details. [1]

OpenShift Pipelines features

Source: [1]

Standard CI/CD pipeline definition based on Tekton
Build images with Kubernetes tools such as S2I, Buildah, Buildpacks, Kaniko, etc
Deploy applications to multiple platforms such as Kubernetes, serverless and VMs
Easy to extend and integrate with existing tools
Scale pipelines on-demand
Portable across any Kubernetes platform
Designed for microservices and decentralized teams
Integrated with the OpenShift Developer Console

Prerequisites

OpenShift 4.x cluster. Try yourself at https://try.openshift.com
Optional: Tekton CLI - Optional for now, since you could do everything via UI as well.
oc/kubectl CLI or WebUI

The Tekton CLI is optional in case you prefer to do everything via the OpenShift WebUI. However, below examples make use of the Tekton CLI and I personally would recommend to at least install it. (Like oc client it is good to have a CLI option as well). In any case, all described action can be done directly via the WebUI as well.

Basic Concepts

Tekton makes use of several custom resources (CRD).

These CRDs are:

Task: each step in a pipeline is a task, while a task can contain several steps itself, which are required to perform a specific task. For each Task a pod will be allocated and for each step inside this Task a container will be used. This helps in better scalability and better performance throughout the pipeline process.
Pipeline: is a series of tasks, combined to work together in a defined (structured) way
TaskRun: is the result of a Task, all combined TaskRuns are used in the PipelineRun
PipelineRun: is the actual execution of a whole Pipeline, containing the results of the pipeline (success, failed…)

Pipelines and Tasks should be generic and never define possible variables, like input git repository, directly in their definition. For this, the concept of PipelineResources has been created, which defines these parameters and which are used during a PipelineRun.

Installation

The OpenShift Pipeline is an operator which can e installed using the following yaml:

cat <<'EOF' > deploy-OpenShift-Pipelines.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: openshift-pipelines-operator
namespace: openshift-operators
spec:
channel: dev-preview
name: openshift-pipelines-operator
source: community-operators
sourceNamespace: openshift-marketplace
EOF

oc create -f deploy-OpenShift-Pipelines.yaml

As an alternative, you can also use the WebUI to rollout the operator:

Search for "OpenShift Pipeline" under OperatorHub and install it
Select:
1. All Namespaces: since the operator needs to watch for Tekton Custom Resources across all namespaces.
2. Channel: Dev Preview
3. Approve Strategy: Automatic

Prepare a tutorial project

To test our OpenShift Pipelines, we need to deploy an example application. This application let’s you vote what pet you like more: Cats or Dogs? It contais of a backend and a frontend part, which both will be deployed in a namespace.

Let’s first create a new project:

oc new-project pipelines-tutorial

The OpenShift Pipeline operator will automatically create a pipeline serviceaccount with all required permissions to build and push an image and which is used by PipelineRuns:

oc get sa pipeline
NAME SECRETS AGE
pipeline 2 15s

Create a Task

A Task is the smallest block of a Pipeline which by itself can contain one or more steps which are executed in order to process a specific element. For each Task a pod is allocated and each step is running in a container inside this pod. Tasks are reusable by other Pipelines. Input and Output specifications can be used to interact with other Tasks.

Let’s create two tasks Source: Pipeline-Tutorial

cat <<'EOF' > deploy-Example-Tasks.yaml
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
name: apply-manifests
spec:
inputs:
resources:
- {type: git, name: source}
params:
- name: manifest_dir
description: The directory in source that contains yaml manifests
type: string
default: "k8s"
steps:
- name: apply
image: quay.io/openshift/origin-cli:latest
workingDir: /workspace/source
command: ["/bin/bash", "-c"]
args:
- |-
echo Applying manifests in $(inputs.params.manifest_dir) directory
oc apply -f $(inputs.params.manifest_dir)
echo -----------------------------------
---
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
name: update-deployment
spec:
inputs:
resources:
- {type: image, name: image}
params:
- name: deployment
description: The name of the deployment patch the image
type: string
steps:
- name: patch
image: quay.io/openshift/origin-cli:latest
command: ["/bin/bash", "-c"]
args:
- |-
oc patch deployment $(inputs.params.deployment) --patch='{"spec":{"template":{"spec":{
"containers":[{
"name": "$(inputs.params.deployment)",
"image":"$(inputs.resources.image.url)"
}]
}}}}'
EOF

oc create -f deploy-Example-Tasks.yaml

Verify that the two tasks have been created using the Tekton CLI:

tkn task ls
NAME AGE
apply-manifests 52 seconds ago
update-deployment 52 seconds ago

Create a Pipeline

A pipeline is a set of Tasks, which should be executed in a defined way to achieve a specific goal.

The example Pipeline below uses two resources:

git-repo: defines the Git-Source
image: Defines the target at a repository

It first uses the Task buildah, which is a standard Task the OpenShift operator created automatically. This task will build the image. The resulted image is pushed to an image registry, defined in the output parameter. After that our created tasks apply-manifest and update-deployment are executed. The execution order of these tasks is defined with the runAfter Parameter in the yaml definition.

The Pipeline should be re-usable accross multiple projects or environments, thats why the resources (git-repo and image) are not defined here. When a Pipeline is executed, these resources will get defined.

cat <<'EOF' > deploy-Example-Pipeline.yaml
apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
name: build-and-deploy
spec:
resources:
- name: git-repo
type: git
- name: image
type: image
params:
- name: deployment-name
type: string
description: name of the deployment to be patched
tasks:
- name: build-image
taskRef:
name: buildah
kind: ClusterTask
resources:
inputs:
- name: source
resource: git-repo
outputs:
- name: image
resource: image
params:
- name: TLSVERIFY
value: "false"
- name: apply-manifests
taskRef:
name: apply-manifests
resources:
inputs:
- name: source
resource: git-repo
runAfter:
- build-image
- name: update-deployment
taskRef:
name: update-deployment
resources:
inputs:
- name: image
resource: image
params:
- name: deployment
value: $(params.deployment-name)
runAfter:
- apply-manifests
EOF

oc create -f deploy-Example-Pipeline.yaml

Verify that the Pipeline has been created using the Tekton CLI:

tkn pipeline ls
NAME AGE LAST RUN STARTED DURATION STATUS
build-and-deploy 3 seconds ago --- --- --- ---

Trigger Pipeline

After the Pipeline has been created, it can be triggered to execute the Tasks.

Create PipelineResources

Since the Pipeline is generic, we need to define 2 PipelineResources first, to execute a Pipepline. Our example application contains a frontend (vote-ui) AND a backend (vote-api), therefore 4 PipelineResources will be created. (2 times git repository to clone the source and 2 time output image)

Quick overview:

ui-repo: will be used as git_repo in the Pipepline for the Frontend
ui-image: will be used as image in the Pipeline for the Frontend
api-repo: will be used as git_repo in the Pipepline for the Backend
api-image: will be used as image in the Pipeline for the Backend

cat <<'EOF' > deploy-Example-PipelineResources.yaml
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: ui-repo
spec:
type: git
params:
- name: url
value: http://github.com/openshift-pipelines/vote-ui.git
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: ui-image
spec:
type: image
params:
- name: url
value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-ui:latest
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: api-repo
spec:
type: git
params:
- name: url
value: http://github.com/openshift-pipelines/vote-api.git
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: api-image
spec:
type: image
params:
- name: url
value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-api:latest
EOF

oc create -f deploy-Example-PipelineResources.yaml

The resources can be listed with:

tkn resource ls
NAME TYPE DETAILS
api-repo git url: http://github.com/openshift-pipelines/vote-api.git
ui-repo git url: http://github.com/openshift-pipelines/vote-ui.git
api-image image url: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-api:latest
ui-image image url: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-ui:latest

Execute Pipelines

We start a PipelineRune for the backend and frontend of our application.

cat <<'EOF' > deploy-Example-PipelineRun.yaml
apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
name: build-deploy-api-pipelinerun
spec:
pipelineRef:
name: build-and-deploy
resources:
- name: git-repo
resourceRef:
name: api-repo
- name: image
resourceRef:
name: api-image
params:
- name: deployment-name
value: vote-api
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
name: build-deploy-ui-pipelinerun
spec:
pipelineRef:
name: build-and-deploy
resources:
- name: git-repo
resourceRef:
name: ui-repo
- name: image
resourceRef:
name: ui-image
params:
- name: deployment-name
value: vote-ui
EOF

oc create -f deploy-Example-PipelineRun.yaml

The PipelineRuns can be listed with

tkn pipelinerun ls
NAME STARTED DURATION STATUS
build-deploy-api-pipelinerun 3 minutes ago --- Running
build-deploy-ui-pipelinerun 3 minutes ago --- Running

Moreover, the logs can be viewed with the following command and select the appropriate PipelineRun:

tkn pipeline logs -f
? Select pipelinerun: [Use arrows to move, type to filter]
> build-deploy-api-pipelinerun started 2 minutes ago
build-deploy-ui-pipelinerun started 2 minutes ago

Checking your application

Now our Pipeline built and deployed the voting application, where you can vote if you prefere cats or dogs (Cats or course :) )

Get the route of your project and open the URL in the browser. (Should be something like vote-ui-pipelines-tutorial.apps.yourclustername)

Figure 1. Tekton: Example Application

OpenShift WebUI

With the OpenShift Pipeline operator a new menu item is introduced on the WebUI of OpenShift. All Tekton CLI command which are used above, can actually be replaced with the web interface, in case you prefere this. The big advantage is th graphical presentation of Pipelines and their lifetime.

I will not create screenshots for every screen, but for example pipelines:

Under Pipelines a list of pipelines will be shown.

Figure 2. OpenShift UI: List of Pipelines

Tekton on TechBlog about OpenShift/Ansible/Satellite and much more