oc compliance command line plugin

- Thomas Jungbauer Thomas Jungbauer ( Lastmod: 2024-05-05 ) - 4 min read

As described at Compliance Operator the Compliance Operator can be used to scan the OpenShift cluster environment against security benchmark, like CIS. Fetching the actual results might be a bit tricky tough.

With OpenShift 4.8 plugins to the oc command are allowed. One of these plugin os oc compliance, which allows you to easily fetch scan results, re-run scans and so on. Let’s install and try it out.

Installation

An oc plugin must be deployed into the same directory as the oc command itself.

The following describes the building and installation of the plugin.

You need Go installed on your node.
  1. Clone the Git repository:

    git clone https://github.com/openshift/oc-compliance.git
  2. Build and install the plugin

    make; make install
    
    go build -o ./bin/oc-compliance ./cmd
    which oc | xargs dirname | xargs -n1 cp ./bin/oc-compliance
  3. The plugin allows the use of oc compliance

    oc compliance
    You must specify a sub-command.
    
    Usage:
      oc-compliance [flags]
      oc-compliance [command]
    
    Available Commands:
      bind        Creates a ScanSettingBinding for the given parameters
      controls    Get a report of what controls you\'re complying with
      fetch-fixes Download the fixes/remediations
      fetch-raw   Download raw compliance results
      help        Help about any command
      rerun-now   Force a re-scan for one or more ComplianceScans
      view-result View a ComplianceCheckResult
    
    Flags:
      -h, --help   help for oc-compliance
    
    Use "oc-compliance [command] --help" for more information about a command.

Fetch Raw Results

Without the oc-compliance plugin it was required to manually spin up a Pod and download the results from this Pod, where the PV is mounted. Now, with a simple command we can select the ScanSettingBinding and define an output folder. For example:

oc compliance fetch-raw <object-type> <object-name> -o <output-path>

Assuming the the compliance operator was configured as in the previous article, we have the ScanSettingBinding called cis-compliance:

oc compliance fetch-raw scansettingbindings cis-compliance -n openshift-compliance -o /tmp/

This starts downloading the result archives into /tmp

Fetching results for cis-compliance scans: ocp4-cis-node-worker, ocp4-cis-node-master, ocp4-cis
Fetching raw compliance results for pod 'raw-result-extractor-fxbw8'.Fetching raw compliance results for scan 'ocp4-cis-node-worker'.........
The raw compliance results are avaliable in the following directory: /tmp/ocp4-cis-node-worker
Fetching raw compliance results for pod 'raw-result-extractor-kqrw5'.Fetching raw compliance results for scan 'ocp4-cis-node-master'.....
The raw compliance results are avaliable in the following directory: /tmp/ocp4-cis-node-master
Fetching raw compliance results for pod 'raw-result-extractor-pfrgk'.Fetching raw compliance results for scan 'ocp4-cis'..
The raw compliance results are avaliable in the following directory: /tmp/ocp4-cis

ls -la /tmp/ocp4-cis*
/tmp/ocp4-cis:
total 172
drwx------   2 root root   4096 Jul 30 16:05 .
drwxrwxrwt. 18 root root   4096 Jul 30 16:05 ..
-rw-r--r--   1 root root 166676 Jul 30 16:05 ocp4-cis-api-checks-pod.xml.bzip2

/tmp/ocp4-cis-node-master:
total 504
drwx------   2 root root   4096 Jul 30 16:05 .
drwxrwxrwt. 18 root root   4096 Jul 30 16:05 ..
-rw-r--r--   1 root root 168256 Jul 30 16:05 ocp4-cis-node-master-master-0-pod.xml.bzip2
-rw-r--r--   1 root root 165716 Jul 30 16:05 ocp4-cis-node-master-master-1-pod.xml.bzip2
-rw-r--r--   1 root root 166945 Jul 30 16:05 ocp4-cis-node-master-master-2-pod.xml.bzip2

/tmp/ocp4-cis-node-worker:
total 1112
drwx------   2 root root   4096 Jul 30 16:05 .
drwxrwxrwt. 18 root root   4096 Jul 30 16:05 ..
-rw-r--r--   1 root root 154943 Jul 30 16:05 ocp4-cis-node-worker-compute-0-pod.xml.bzip2
-rw-r--r--   1 root root 154903 Jul 30 16:05 ocp4-cis-node-worker-compute-1-pod.xml.bzip2
-rw-r--r--   1 root root 154939 Jul 30 16:05 ocp4-cis-node-worker-compute-2-pod.xml.bzip2
-rw-r--r--   1 root root 154890 Jul 30 16:05 ocp4-cis-node-worker-compute-3-pod.xml.bzip2
-rw-r--r--   1 root root 168175 Jul 30 16:05 ocp4-cis-node-worker-master-0-pod.xml.bzip2
-rw-r--r--   1 root root 165603 Jul 30 16:05 ocp4-cis-node-worker-master-1-pod.xml.bzip2
-rw-r--r--   1 root root 166914 Jul 30 16:05 ocp4-cis-node-worker-master-2-pod.xml.bzip2

Re-Run Scans

Sometimes it is necessary to re-run scans. This can be done by annotating the appropriate scan as described at: Performing a Rescan

With the oc plugin you can simply trigger a re-scan with a single command:

oc compliance rerun-now scansettingbindings <name of scanbinding>

For example:

oc compliance rerun-now scansettingbindings cis-compliance

Example output:

Rerunning scans from 'cis-compliance': ocp4-cis-node-worker, ocp4-cis-node-master, ocp4-cis
Re-running scan 'openshift-compliance/ocp4-cis-node-worker'
Re-running scan 'openshift-compliance/ocp4-cis-node-master'
Re-running scan 'openshift-compliance/ocp4-cis'

With the command oc get compliancescan -n openshift-compliance you can check when the scan has been done:

NAME                   PHASE         RESULT
ocp4-cis               RUNNING       NOT-AVAILABLE
ocp4-cis-node-master   RUNNING       NOT-AVAILABLE
ocp4-cis-node-worker   AGGREGATING   NOT-AVAILABLE

View Results on CLI

Once a scan process has finished you can verify the check results quick and easy using the command line:

oc get ComplianceCheckResult -A

This prints for example:

NAMESPACE              NAME                                                                           STATUS           SEVERITY
[...]
openshift-compliance   ocp4-cis-audit-log-forwarding-enabled                                          FAIL             medium
[...]

The view-result can print a human readable output, for example:

oc compliance view-result ocp4-cis-audit-log-forwarding-enabled -n openshift-compliance

Example:

+----------------------+-----------------------------------------------------------------------------------------+
|         KEY          |                                          VALUE                                          |
+----------------------+-----------------------------------------------------------------------------------------+
| Title                | Ensure that Audit Log                                                                   |
|                      | Forwarding Is Enabled                                                                   |
+----------------------+-----------------------------------------------------------------------------------------+
| Status               | FAIL                                                                                    |
+----------------------+-----------------------------------------------------------------------------------------+
| Severity             | medium                                                                                  |
+----------------------+-----------------------------------------------------------------------------------------+
| Description          | OpenShift audit works at the                                                            |
|                      | API server level, logging                                                               |
|                      | all requests coming to the                                                              |
|                      | server. Audit is on by default                                                          |
|                      | and the best practice is                                                                |
|                      | to ship audit logs off the                                                              |
|                      | cluster for retention. The                                                              |
|                      | cluster-logging-operator is                                                             |
|                      | able to do this with the                                                                |
|                      |                                                                                         |
|                      |                                                                                         |
|                      |                                                                                         |
|                      | ClusterLogForwarders                                                                    |
|                      |                                                                                         |
|                      |                                                                                         |
|                      |                                                                                         |
|                      | resource. The forementioned resource can be configured to logs to different third party |
|                      | systems. For more information on this, please reference the official documentation:     |
|                      | https://docs.openshift.com/container-platform/4.6/logging/cluster-logging-external.html |
+----------------------+-----------------------------------------------------------------------------------------+
| Rationale            | Retaining logs ensures the                                                              |
|                      | ability to go back in time to                                                           |
|                      | investigate or correlate any                                                            |
|                      | events. Offloading audit logs                                                           |
|                      | from the cluster ensures that                                                           |
|                      | an attacker that has access                                                             |
|                      | to the cluster will not be                                                              |
|                      | able to tamper with the logs                                                            |
|                      | because of the logs being                                                               |
|                      | stored off-site.                                                                        |
+----------------------+-----------------------------------------------------------------------------------------+
| Instructions         | Run the following command:                                                              |
|                      |                                                                                         |
|                      | oc get clusterlogforwarders                                                             |
|                      | instance -n openshift-logging                                                           |
|                      | -ojson | jq -r                                                                          |
|                      | '.spec.pipelines[].inputRefs |                                                          |
|                      | contains(["audit"])'                                                                    |
|                      |                                                                                         |
|                      | The output should return true.                                                          |
+----------------------+-----------------------------------------------------------------------------------------+
| CIS-OCP Controls     | 1.2.23                                                                                  |
+----------------------+-----------------------------------------------------------------------------------------+
| NIST-800-53 Controls | AC-2(12), AU-6, AU-6(1),                                                                |
|                      | AU-6(3), AU-9(2), SI-4(16),                                                             |
|                      | AU-4(1), AU-11, AU-7, AU-7(1)                                                           |
+----------------------+-----------------------------------------------------------------------------------------+
| Available Fix        | No                                                                                      |
+----------------------+-----------------------------------------------------------------------------------------+
| Result Object Name   | ocp4-cis-audit-log-forwarding-enabled                                                   |
+----------------------+-----------------------------------------------------------------------------------------+
| Rule Object Name     | ocp4-audit-log-forwarding-enabled                                                       |
+----------------------+-----------------------------------------------------------------------------------------+
| Remediation Created  | No                                                                                      |
+----------------------+-----------------------------------------------------------------------------------------+