Introduction

OpenShift Pod Placement

- Thomas Jungbauer Thomas Jungbauer ( Lastmod: 2024-05-08 ) - 2 min read

Pod scheduling is an internal process that determines placement of new pods onto nodes within the cluster. It is probably one of the most important tasks for a Day-2 scenario and should be considered at a very early stage for a new cluster. OpenShift/Kubernetes is already shipped with a default scheduler which schedules pods as they get created accross the cluster, without any manual steps.

However, there are scenarios where a more advanced approach is required, like for example using a specifc group of nodes for dedicated workload or make sure that certain applications do not run on the same nodes. Kubernetes provides different options:

  • Controlling placement with node selectors

  • Controlling placement with pod/node affinity/anti-affinity rules

  • Controlling placement with taints and tolerations

  • Controlling placement with topology spread constraints

This series will try to go into the detail of the different options and explains in simple examples how to work with pod placement rules. It is not a replacement for any official documentation, so always check out Kubernetes and or OpenShift documentations.

Pod Placement Series

  1. NodeSelector

  2. Pod Affinity and Anti Affinity

  3. Node Affinity

  4. Taints and Tolerations

  5. Topology Spread Constraints

  6. Descheduler

Prerequisites

The following prerequisites are used for all examples.

Let’s image that our cluster (OpenShift 4) has 4 compute nodes

oc get node --selector='node-role.kubernetes.io/worker'

NAME        STATUS   ROLES           AGE     VERSION
compute-0   Ready    worker          7h1m    v1.19.0+d59ce34
compute-1   Ready    worker          7h1m    v1.19.0+d59ce34
compute-2   Ready    worker          7h1m    v1.19.0+d59ce34
compute-3   Ready    worker          7h1m    v1.19.0+d59ce34

An example application (from the catalog Django + Postgres) has been deployed in the namespace podtesting. It contains by default 1 pod for a PostGresql database and one pod for a frontend web application.

oc get pods -n podtesting -o wide | grep Running
django-psql-example-1-h6kst    1/1     Running     0          20m   10.130.2.97   compute-2   <none>           <none>
postgresql-1-4pcm4             1/1     Running     0          21m   10.131.0.51   compute-3   <none>           <none>

Without any configuration the OpenShift scheduler will try to spread the pods evenly accross the cluster.

Let’s increase the replica of the web frontend:

oc scale --replicas=4 dc/django-psql-example -n podtesting

Eventually 4 additional pods will be started accross the compute nodes of the cluster:

oc get pods -n podtesting -o wide | grep Running

django-psql-example-1-842fl    1/1     Running             0          2m7s   10.131.0.65   compute-3   <none>           <none>
django-psql-example-1-h6kst    1/1     Running             0          24m    10.130.2.97   compute-2   <none>           <none>
django-psql-example-1-pxhlv    1/1     Running             0          2m7s   10.128.2.13   compute-0   <none>           <none>
django-psql-example-1-xms7x    1/1     Running             0          2m7s   10.129.2.10   compute-1   <none>           <none>
postgresql-1-4pcm4             1/1     Running             0          26m    10.131.0.51   compute-3   <none>           <none>

As you can see, the scheduler already tries to spread the pods evenly, so that every worker node will host one frontend pod.

However, let’s try to apply a more advanced configuration for the pod placement, starting with the Pod Placement - NodeSelector

See Also:

  • Single log out from Keycloak and OpenShift –

    The following 1-minute article is a follow-up to my previous article about how to use Keycloak as an authentication provider for OpenShift. In this article, I will show you how to configure Keycloak and OpenShift for Single Log Out (SLO). This means that when you log out from Keycloak, you will also be logged out from OpenShift automatically. This requires some additional configuration in Keycloak and OpenShift, but it is not too complicated.

  • Step by Step - Using Keycloak Authentication in OpenShift –

    I was recently asked about how to use Keycloak as an authentication provider for OpenShift. How to install Keycloak using the Operator and how to configure Keycloak and OpenShift so that users can log in to OpenShift using OpenID. I have to admit that the exact steps are not easy to find, so I decided to write a blog post about it, describing each step in detail. This time I will not use GitOps, but the OpenShift and Keycloak Web Console to show the steps, because before we put it into GitOps, we need to understand what is actually happening. This article tries to explain every step required so that a user can authenticate to OpenShift using Keycloak as an Identity Provider (IDP) and that Groups from Keycloak are imported into OpenShift. This article does not cover a production grade installation of Keycloak, but only a test installation, so you can see how it works. For production, you might want to consider a proper database (maybe external, but at least with a backup), high availability, etc.).

  • Using ApplicationSet with Matrix Generator and define individual Namespaces –

    During my day-to-day business, I am discussing the following setup with many customers: Configure App-of-Apps. Here I try to explain how I use an ApplicationSet that watches over a folder in Git and automatically adds a new Argo CD Application whenever a new folder is found. This works great, but there is a catch: The ApplicationSet uses the same Namespace default for all Applications. This is not always desired, especially when you have different teams working on different Applications. Recently I was asked by the customer if this can be fixed and if it is possible to define different Namespaces for each Application. The answer is yes, and I would like to show you how to do this.

  • Introducing AdminNetworkPolicies –

    Classic Kubernetes/OpenShift offer a feature called NetworkPolicy that allows users to control the traffic to and from their assigned Namespace. NetworkPolicies are designed to give project owners or tenants the ability to protect their own namespace. Sometimes, however, I worked with customers where the cluster administrators or a dedicated (network) team need to enforce these policies. Since the NetworkPolicy API is namespace-scoped, it is not possible to enforce policies across namespaces. The only solution was to create custom (project) admin and edit roles, and remove the ability of creating, modifying or deleting NetworkPolicy objects. Technically, this is possible and easily done. But shifts the whole network security to cluster administrators. Luckily, this is where AdminNetworkPolicy (ANP) and BaselineAdminNetworkPolicy (BANP) comes into play.

  • Using Kustomize to post render a Helm Chart –

    Lately I came across several issues where a given Helm Chart must be modified after it has been rendered by Argo CD. Argo CD does a helm template to render a Chart. Sometimes, especially when you work with Subcharts or when a specific setting is not yet supported by the Chart, you need to modify it later …​ you need to post-render the Chart. In this very short article, I would like to demonstrate this on a real-live example I had to do. I would like to inject annotations to a Route objects, so that the certificate can be injected. This is done by the cert-utils operator. For the post-rendering the Argo CD repo pod will be extended with a sidecar container, that is watching for the repos and patches them if required.

Copyright © 2020 - 2025 Toni Schmidbauer & Thomas Jungbauer