oc compliance command line plugin

OpenShift Compliance Security

- Thomas Jungbauer Thomas Jungbauer ( Lastmod: 2024-05-05 ) - 4 min read

As described at Compliance Operator the Compliance Operator can be used to scan the OpenShift cluster environment against security benchmark, like CIS. Fetching the actual results might be a bit tricky tough.

With OpenShift 4.8 plugins to the oc command are allowed. One of these plugin os oc compliance, which allows you to easily fetch scan results, re-run scans and so on. Let’s install and try it out.

Installation

An oc plugin must be deployed into the same directory as the oc command itself.

The following describes the building and installation of the plugin.

You need Go installed on your node.

  1. Clone the Git repository:

    git clone https://github.com/openshift/oc-compliance.git

  2. Build and install the plugin

    make; make install

go build -o ./bin/oc-compliance ./cmd
which oc | xargs dirname | xargs -n1 cp ./bin/oc-compliance

  3. The plugin allows the use of oc compliance

    oc compliance
You must specify a sub-command.

Usage:
  oc-compliance [flags]
  oc-compliance [command]

Available Commands:
  bind        Creates a ScanSettingBinding for the given parameters
  controls    Get a report of what controls you\'re complying with
  fetch-fixes Download the fixes/remediations
  fetch-raw   Download raw compliance results
  help        Help about any command
  rerun-now   Force a re-scan for one or more ComplianceScans
  view-result View a ComplianceCheckResult

Flags:
  -h, --help   help for oc-compliance

Use "oc-compliance [command] --help" for more information about a command.

Fetch Raw Results

Without the oc-compliance plugin it was required to manually spin up a Pod and download the results from this Pod, where the PV is mounted. Now, with a simple command we can select the ScanSettingBinding and define an output folder. For example:

oc compliance fetch-raw <object-type> <object-name> -o <output-path>

Assuming the the compliance operator was configured as in the previous article, we have the ScanSettingBinding called cis-compliance:

oc compliance fetch-raw scansettingbindings cis-compliance -n openshift-compliance -o /tmp/

This starts downloading the result archives into /tmp

Fetching results for cis-compliance scans: ocp4-cis-node-worker, ocp4-cis-node-master, ocp4-cis
Fetching raw compliance results for pod 'raw-result-extractor-fxbw8'.Fetching raw compliance results for scan 'ocp4-cis-node-worker'.........
The raw compliance results are avaliable in the following directory: /tmp/ocp4-cis-node-worker
Fetching raw compliance results for pod 'raw-result-extractor-kqrw5'.Fetching raw compliance results for scan 'ocp4-cis-node-master'.....
The raw compliance results are avaliable in the following directory: /tmp/ocp4-cis-node-master
Fetching raw compliance results for pod 'raw-result-extractor-pfrgk'.Fetching raw compliance results for scan 'ocp4-cis'..
The raw compliance results are avaliable in the following directory: /tmp/ocp4-cis

ls -la /tmp/ocp4-cis*
/tmp/ocp4-cis:
total 172
drwx------   2 root root   4096 Jul 30 16:05 .
drwxrwxrwt. 18 root root   4096 Jul 30 16:05 ..
-rw-r--r--   1 root root 166676 Jul 30 16:05 ocp4-cis-api-checks-pod.xml.bzip2

/tmp/ocp4-cis-node-master:
total 504
drwx------   2 root root   4096 Jul 30 16:05 .
drwxrwxrwt. 18 root root   4096 Jul 30 16:05 ..
-rw-r--r--   1 root root 168256 Jul 30 16:05 ocp4-cis-node-master-master-0-pod.xml.bzip2
-rw-r--r--   1 root root 165716 Jul 30 16:05 ocp4-cis-node-master-master-1-pod.xml.bzip2
-rw-r--r--   1 root root 166945 Jul 30 16:05 ocp4-cis-node-master-master-2-pod.xml.bzip2

/tmp/ocp4-cis-node-worker:
total 1112
drwx------   2 root root   4096 Jul 30 16:05 .
drwxrwxrwt. 18 root root   4096 Jul 30 16:05 ..
-rw-r--r--   1 root root 154943 Jul 30 16:05 ocp4-cis-node-worker-compute-0-pod.xml.bzip2
-rw-r--r--   1 root root 154903 Jul 30 16:05 ocp4-cis-node-worker-compute-1-pod.xml.bzip2
-rw-r--r--   1 root root 154939 Jul 30 16:05 ocp4-cis-node-worker-compute-2-pod.xml.bzip2
-rw-r--r--   1 root root 154890 Jul 30 16:05 ocp4-cis-node-worker-compute-3-pod.xml.bzip2
-rw-r--r--   1 root root 168175 Jul 30 16:05 ocp4-cis-node-worker-master-0-pod.xml.bzip2
-rw-r--r--   1 root root 165603 Jul 30 16:05 ocp4-cis-node-worker-master-1-pod.xml.bzip2
-rw-r--r--   1 root root 166914 Jul 30 16:05 ocp4-cis-node-worker-master-2-pod.xml.bzip2

Re-Run Scans

Sometimes it is necessary to re-run scans. This can be done by annotating the appropriate scan as described at: Performing a Rescan

With the oc plugin you can simply trigger a re-scan with a single command:

oc compliance rerun-now scansettingbindings <name of scanbinding>

For example:

oc compliance rerun-now scansettingbindings cis-compliance

Example output:

Rerunning scans from 'cis-compliance': ocp4-cis-node-worker, ocp4-cis-node-master, ocp4-cis
Re-running scan 'openshift-compliance/ocp4-cis-node-worker'
Re-running scan 'openshift-compliance/ocp4-cis-node-master'
Re-running scan 'openshift-compliance/ocp4-cis'

With the command oc get compliancescan -n openshift-compliance you can check when the scan has been done:

NAME                   PHASE         RESULT
ocp4-cis               RUNNING       NOT-AVAILABLE
ocp4-cis-node-master   RUNNING       NOT-AVAILABLE
ocp4-cis-node-worker   AGGREGATING   NOT-AVAILABLE

View Results on CLI

Once a scan process has finished you can verify the check results quick and easy using the command line:

oc get ComplianceCheckResult -A

This prints for example:

NAMESPACE              NAME                                                                           STATUS           SEVERITY
[...]
openshift-compliance   ocp4-cis-audit-log-forwarding-enabled                                          FAIL             medium
[...]

The view-result can print a human readable output, for example:

oc compliance view-result ocp4-cis-audit-log-forwarding-enabled -n openshift-compliance

Example:

+----------------------+-----------------------------------------------------------------------------------------+
|         KEY          |                                          VALUE                                          |
+----------------------+-----------------------------------------------------------------------------------------+
| Title                | Ensure that Audit Log                                                                   |
|                      | Forwarding Is Enabled                                                                   |
+----------------------+-----------------------------------------------------------------------------------------+
| Status               | FAIL                                                                                    |
+----------------------+-----------------------------------------------------------------------------------------+
| Severity             | medium                                                                                  |
+----------------------+-----------------------------------------------------------------------------------------+
| Description          | OpenShift audit works at the                                                            |
|                      | API server level, logging                                                               |
|                      | all requests coming to the                                                              |
|                      | server. Audit is on by default                                                          |
|                      | and the best practice is                                                                |
|                      | to ship audit logs off the                                                              |
|                      | cluster for retention. The                                                              |
|                      | cluster-logging-operator is                                                             |
|                      | able to do this with the                                                                |
|                      |                                                                                         |
|                      |                                                                                         |
|                      |                                                                                         |
|                      | ClusterLogForwarders                                                                    |
|                      |                                                                                         |
|                      |                                                                                         |
|                      |                                                                                         |
|                      | resource. The forementioned resource can be configured to logs to different third party |
|                      | systems. For more information on this, please reference the official documentation:     |
|                      | https://docs.openshift.com/container-platform/4.6/logging/cluster-logging-external.html |
+----------------------+-----------------------------------------------------------------------------------------+
| Rationale            | Retaining logs ensures the                                                              |
|                      | ability to go back in time to                                                           |
|                      | investigate or correlate any                                                            |
|                      | events. Offloading audit logs                                                           |
|                      | from the cluster ensures that                                                           |
|                      | an attacker that has access                                                             |
|                      | to the cluster will not be                                                              |
|                      | able to tamper with the logs                                                            |
|                      | because of the logs being                                                               |
|                      | stored off-site.                                                                        |
+----------------------+-----------------------------------------------------------------------------------------+
| Instructions         | Run the following command:                                                              |
|                      |                                                                                         |
|                      | oc get clusterlogforwarders                                                             |
|                      | instance -n openshift-logging                                                           |
|                      | -ojson | jq -r                                                                          |
|                      | '.spec.pipelines[].inputRefs |                                                          |
|                      | contains(["audit"])'                                                                    |
|                      |                                                                                         |
|                      | The output should return true.                                                          |
+----------------------+-----------------------------------------------------------------------------------------+
| CIS-OCP Controls     | 1.2.23                                                                                  |
+----------------------+-----------------------------------------------------------------------------------------+
| NIST-800-53 Controls | AC-2(12), AU-6, AU-6(1),                                                                |
|                      | AU-6(3), AU-9(2), SI-4(16),                                                             |
|                      | AU-4(1), AU-11, AU-7, AU-7(1)                                                           |
+----------------------+-----------------------------------------------------------------------------------------+
| Available Fix        | No                                                                                      |
+----------------------+-----------------------------------------------------------------------------------------+
| Result Object Name   | ocp4-cis-audit-log-forwarding-enabled                                                   |
+----------------------+-----------------------------------------------------------------------------------------+
| Rule Object Name     | ocp4-audit-log-forwarding-enabled                                                       |
+----------------------+-----------------------------------------------------------------------------------------+
| Remediation Created  | No                                                                                      |
+----------------------+-----------------------------------------------------------------------------------------+

See Also:

  • Introducing AdminNetworkPolicies –

    Classic Kubernetes/OpenShift offer a feature called NetworkPolicy that allows users to control the traffic to and from their assigned Namespace. NetworkPolicies are designed to give project owners or tenants the ability to protect their own namespace. Sometimes, however, I worked with customers where the cluster administrators or a dedicated (network) team need to enforce these policies. Since the NetworkPolicy API is namespace-scoped, it is not possible to enforce policies across namespaces. The only solution was to create custom (project) admin and edit roles, and remove the ability of creating, modifying or deleting NetworkPolicy objects. Technically, this is possible and easily done. But shifts the whole network security to cluster administrators. Luckily, this is where AdminNetworkPolicy (ANP) and BaselineAdminNetworkPolicy (BANP) comes into play.

  • Using Kustomize to post render a Helm Chart –

    Lately I came across several issues where a given Helm Chart must be modified after it has been rendered by Argo CD. Argo CD does a helm template to render a Chart. Sometimes, especially when you work with Subcharts or when a specific setting is not yet supported by the Chart, you need to modify it later …​ you need to post-render the Chart. In this very short article, I would like to demonstrate this on a real-live example I had to do. I would like to inject annotations to a Route objects, so that the certificate can be injected. This is done by the cert-utils operator. For the post-rendering the Argo CD repo pod will be extended with a sidecar container, that is watching for the repos and patches them if required.

  • Managing Certificates using GitOps approach –

    The article SSL Certificate Management for OpenShift on AWS explains how to use the Cert-Manager Operator to request and install a new SSL Certificate. This time, I would like to leverage the GitOps approach using the Helm Chart cert-manager I have prepared to deploy the Operator and order new Certificates. I will use an ACME Letsencrypt issuer with a DNS challenge. My domain is hosted at AWS Route 53. However, any other integration can be easily used.

  • Update Cluster Version using GitOps approach –

    During a GitOps journey at one point, the question arises, how to update a cluster? Nowadays it is very easy to update a cluster using CLI or WebUI, so why bother with GitOps in that case? The reason is simple: Using GitOps you can be sure that all clusters are updated to the correct, required version and the version of each cluster is also managed in Git. All you need is the channel you want to use and the desired cluster version. Optionally, you can define the exact image SHA. This might be required when you are operating in a restricted environment.

  • Multiple Sources for Applications in Argo CD –

    Argo CD or OpenShift GitOps uses Applications or ApplicationSets to define the relationship between a source (Git) and a cluster. Typically, this is a 1:1 link, which means one Application is using one source to compare the cluster status. This can be a limitation. For example, if you are working with Helm Charts and a Helm repository, you do not want to re-build (or re-release) the whole chart just because you made a small change in the values file that is packaged into the repository. You want to separate the configuration of the chart with the Helm package. The most common scenarios for multiple sources are (see: Argo CD documentation): Your organization wants to use an external/public Helm chart You want to override the Helm values with your own local values You don’t want to clone the Helm chart locally as well because that would lead to duplication and you would need to monitor it manually for upstream changes. This small article describes three different ways with a working example and tries to cover the advantages and disadvantages of each of them. They might be opinionated but some of them proved to be easier to use and manage.

Copyright © 2020 - 2024 Toni Schmidbauer & Thomas Jungbauer