Summary: Secure Supply Chain
Introduction to a Secure Supply Chain
The goal of the following ("short") series is to build a secure CI/CD pipeline step by step using OpenShift Pipelines (based on Tekton). The whole build process shall pull and build an image, upload it to a development environment and subsequently update the production environment.
The main focus here is security. Several steps and tools shall help to build and deploy a Secure Supply Chain.
The whole process is part of a Red Hat workshop which can present to your organization. I did some tweaks and created a step-by-step plan in order to remember it … since I am getting old :)
Step 1 - Listen to Events
In this first step, we will simply prepare our environment to be able to retrieve calls from Git. In Git we will fork a prepared source code into a repository and any time a developer pushes a new code into our repository a webhook will notify OpenShift Pipelines to start the pipeline. Like most pipelines, the first task to be executed is to fetch the source code so it can be used for the next steps. The application I am going to use is called globex-ui and is an example webUI build with Angular.
Step 2 - Pipelines
We will now create the Pipeline and try to trigger it for the first time to verify if our Webhook works as intended.
Step 3 - SonarQube
After the Pipeline has been created and tested we will add another Task to verify the source code and check for possible security issues, leveraging the tool SonarQube by Sonar.
Step 4 - Verify Git Commit
Besides checking the source code quality, we should also verify if the commit into Git was done by someone/something we trust. It is a good practice to sign all commits to Git. You need to prepare your Git account and create trusted certificates.
Step 5 - Build and Sign Image
Finally, after pulling and checking the code, we are going to create the image. During this process the image will be signed and uploaded to the public registry Quay.io.
Step 6 - Scanning with ACS
In this step we will install Advanced Cluster Security (ACS) and create 2 new steps in our Pipeline to scan the image for vulnerabilities and security policy. A custom security policy, configured in ACS, will verify if the image is signed.
Step 7 - Generating a SBOM
A software bill of materials (SBOM) offers an insight into the makeup of an application developed using third-party commercial tools and open-source software. It is a machine-readable, formally structured list of all components. This list is important to gain visibility into what risks might exist in a package and their potential impact. It allows to prioritize the risks and define a remediation path. The teams can use the SBOM to monitor all components that are used across an application.
We will create a Task that will generate a SBOM and uploads it into an SBOM repository.
Step 8 - Updating Kubernetes Manifests
With the finalization of the build process and the security checks during this phase, it is now time to update the Kubernetes manifests and provide the new tag for the created image. I have forked (and cleaned up) another repository that will store all yaml specifications that are required for OpenShift. You can find this repository at: Kubernetes Manifests
Step 9 - Linting Kubernetes Manifests
At this point we have checked our source code, verified that it has been signed and has no vulnerabilities, generated a SBOM and updated the Kubernetes manifests, that are responsible to deploy our application on OpenShift. As everything with OpenShift these Kubernetes objects are simple yaml files. In this step we will perform a linting on these files, to verify if they follow certain rules and best practices.
Step 10 - The Example Application
If you read all articles up to here (congratulations) you know that we always update the README file of "The Application". But what is this application exactly and how can we update it like in a real-life example? The Globex UI application is built with Angular and was prepared for this journey. It is a quite complex application and requires Kafka to be installed as well. However, since I am not a developer and this tool was already available, I forked and re-used it. The original can be found at Globex UI
Step 11 - ACS Deployment Check
After the Pipeline prepared the new image for DEV it should be checked against ACS for policy compliance. This ensures that the deployment manifest adheres to policy requirements. The command line tool roxctl will be leveraged to perform this task.
Step 12 - Verify TLog Signature
Since the image has been deployed on DEV now, we need to prepare everything for production. Before we start, we need to confirm that the whole signing process has been passed and that our signature has been applied correctly. With CoSign we signed our image and used Rekor to create a transparency log. We will use this log to confirm that the image was signed.
Step 13 - Bring it to Production
If you reached this article, congratulations! You read through tons of pages to build up a Pipeline. The last two steps in our Pipeline are: Creating a new branch and creating a pull request, with the changes of the image tag that must be approved and will be merged then (We will not update the main branch directly!). Finally, we will do a "real" update to the application to see the actual changes.
Copyright © 2020 - 2024 Toni Schmidbauer & Thomas Jungbauer