Stumbling into Azure Part II: Setting up a private ARO cluster

- By: Toni Schmidbauer ( Lastmod: 2021-11-16 ) - 2 min read

In Part I of our blog post we covered setting up required resources in Azure. Now we are finally going to set up a private cluster. Private

As review from Part I here is our planned setup, this time including the ARO cluster.

Azure Setup

The diagram below depicts our planned setup:

/Azure/images/azure_network_setup_with_aro.png

On the right hand side can see the resources required for our lab:

  • a virtual network (vnet 192.168.128.0/19). This vnet will be split into 3 separate subnets
  • a master subnet (192.168.129.0/24) holding the ARO control plane nodes
  • a node subnet (192.168.130.0/24) holding ARO worker nodes
  • and finally a subnet call GatewaySubnet where we are going to deploy our Azure VPN gateway (called a vnet-gateway)

    The subnet where the Azure VPN gateway is located needs to have the name GatewaySubnet. Otherwise creating the Azure VPN gateway will fail.

  • we also need a publicIP resource that we are going to connect to our vnet-gateway (the VPN gateway)
  • and finally a local-gateway resource that tells the vnet-gateway which networks are reachable on the left, in our case the Hetzner server.

Creating the private Azure Red Hat OpenShift cluster

  1. Register required resource providers

    az provider register -n Microsoft.RedHatOpenShift --wait
az provider register -n Microsoft.Compute --wait
az provider register -n Microsoft.Storage --wait
az provider register -n Microsoft.Authorization --wait

  1. First we are going to set some environment variable. Those variables are used in the upcoming commands:

    export RESOURCEGROUP=aro-rg
export CLUSTER="aro1"
export GATWAY_SUBNET="192.168.128.0/24"
export MASTER_SUBNET="192.168.129.0/24"
export WORKER_SUBNET="192.168.130.0/24"
export HETZNER_VM_NETWORKS="10.0.0.0/24 192.168.122.0/24 172.16.100.0/24"

  2. Disable subnet private endpoint policies

    az network vnet subnet update \
--name master-subnet \
--resource-group $RESOURCEGROUP \
--vnet-name aro-vnet \
--disable-private-link-service-network-policies true

  3. Create a private DNS zone for our cluster

    az network private-dns zone create -n private2.tntinfra.net -g aro-rg

  4. Create the cluster

    az aro create \
--resource-group $RESOURCEGROUP \
--name $CLUSTER \
--vnet aro-vnet \
--master-subnet master-subnet \
--worker-subnet worker-subnet \
--apiserver-visibility Private \
--ingress-visibility Private \
--domain private.tntinfra.net
# --pull-secret @pull-secret.txt # [OPTIONAL]

  5. After successful cluster creating add DNS entry for the API and Ingress

    Query the Azure API for the API server IP and the ingress IP addresses:

    az aro show -n aro1 -g aro-rg --query '{api:apiserverProfile.ip, ingress:ingressProfiles[0].ip}'

    Example output

    Api            Ingress
-------------  ---------------
192.168.129.4  192.168.130.254

    Add entries to Azure private DNS

    az network private-dns record-set a add-record -g aro-rg -z private.tntinfra.net -a "192.168.129.4" -n api
az network private-dns record-set a add-record -g aro-rg -z private.tntinfra.net -a "192.168.130.254" -n "*.apps"

    List entries to verify configuration

    az network private-dns record-set a list -g aro-rg -z private.tntinfra.net

    Output:

    Name    ResourceGroup    Ttl    Type    AutoRegistered    Metadata
------  ---------------  -----  ------  ----------------  ----------
api     aro-rg           3600   A       False
*.apps  aro-rg           3600   A       False

  6. List cluster credentials after successful setup

    az aro list-credentials \
--name $CLUSTER \
--resource-group $RESOURCEGROUP

  7. Get the console URL

    az aro show \
--name $CLUSTER \
--resource-group $RESOURCEGROUP \
--query "consoleProfile.url" -o tsv

DNS, curl

this works, dunno why?

dig @192.168.129.7 console-openshift-console.apps.xm7rdz4r.westeurope.aroapp.io

use curl to access the internal API and see if it works:

curl -kv https://192.168.129.4:6443

Copyright © 2020 - 2023 Toni Schmidbauer & Thomas Jungbauer

Built with Hugo Learn Theme and Hugo