The Hitchhiker's Guide to Observability - Limit Read Access to Traces - Part 8

OpenShift Observability Distributed Tracing OpenTelemetry Tempo Grafana

- Thomas Jungbauer Thomas Jungbauer ( Lastmod: 2025-12-06 ) - 3 min read

image from The Hitchhiker's Guide to Observability - Limit Read Access to Traces - Part 8

In the previous articles, we deployed a distributed tracing infrastructure with TempoStack and OpenTelemetry Collector. We also deployed a Grafana instance to visualize the traces. The configuration was done in a way that allows everybody to read the traces. Every system:authenticated user is able to read ALL traces. This is usually not what you want. You want to limit trace access to only the appropriate namespace.

In this article, we’ll limit the read access to traces. The users of the team-a namespace will only be able to see their own traces.

Prerequisites

Before we begin, make sure you have:

  • TempoStack deployed and configured (from Part 2)

  • Team-a namespace with traces flowing (from Part 4)

  • A separate user for the team-a namespace.

Verify Trace Access

Let’s verify what a user can see when they are authenticated. We have user1 who is a member of the team-a namespace. Let’s log in as this user and verify what they can see.

Navigate to Observability > Traces and select the tempostack/simplest datasource:

Observability Traces Team-a

You should see the traces for the team-a namespace. This is fine—that’s what we want.

But now let’s change the tenant to tenantB and verify what the user can see.

Observability Traces Team-b

As you can see, the user can see traces from the tenantB namespace, although they are a member of the team-a namespace. This is not what we want. We want to limit trace access to only the appropriate namespace.

The Original RBAC Configuration

In Part 2, we created a ClusterRoleBinding to grant read access to traces for everybody who is authenticated.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tempostack-traces-reader
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: 'system:authenticated'
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: tempostack-traces-reader

This ClusterRoleBinding allows system:authenticated users to read all traces from all tenants and is bound to the ClusterRole tempostack-traces-reader.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tempostack-traces-reader
rules:
  - verbs:
      - get
    apiGroups:
      - tempo.grafana.com
    resources:
      - tenantA
      - tenantB
    resourceNames:
      - traces

This is the configuration we want to change. We want to limit read access to traces to only the appropriate namespace.

Change the RBAC Configuration

We will change the RBAC configuration to limit read access to traces. To do so, we will first create a new ClusterRole for the team-a namespace and bind it to users of that namespace.

Create the new ClusterRole:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tempostack-traces-reader-team-a (1)
rules:
  - verbs:
      - get
    apiGroups:
      - tempo.grafana.com
    resources:
      - tenantA (2)
    resourceNames:
      - traces
1Name of the ClusterRole, now with the prefix team-a.
2The Tenant that is allowed to read the traces. This time it is only the tenantA tenant.

Create the new ClusterRoleBinding:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tempostack-traces-reader-team-a (1)
subjects:
  - kind: User (2)
    apiGroup: rbac.authorization.k8s.io
    name: user1
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: tempostack-traces-reader-team-a (3)
1Name of the ClusterRoleBinding, now with the prefix team-a.
2The User that is allowed to read the traces. In this example: user1.
3The ClusterRole that is allowed to read the traces.
In this example, we are using a single user. In a real-world scenario, you would most likely have a group of users. In that case, you would use a Group instead of a User.

Modify the Original ClusterRole:

As a final step, we need to modify the original ClusterRole to remove tenantB from the list of tenants that are allowed to read the traces.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tempostack-traces-reader
rules:
  - verbs:
      - get
    apiGroups:
      - tempo.grafana.com
    resources:
      - tenantB (1)
    resourceNames:
      - traces
1Only tenantA remains. Remove tenantB from the list of resources.

This removes the permission to read traces from the tenantB namespace for the system:authenticated group.

Eventually, the original ClusterRoleBinding might be deleted once every user has been assigned to a separate ClusterRole.

Verify the Changes

Let’s see what these changes do. As user1, you should still be able to see the traces from the tenantA namespace.

Observability Traces Team-a for user1

But if you change the tenant to tenantB, you should see an error message like this:

Observability Traces Team-b for user1
The user will still see the list of tenants. Hopefully, this will be fixed in a future version of TempoStack.
The Hitchhiker's Guide to Observability Part 8 of 8
1 The Hitchhiker's Guide to Observability … 2 The Hitchhiker's Guide to Observability … 3 The Hitchhiker's Guide to Observability … 4 The Hitchhiker's Guide to Observability … 5 The Hitchhiker's Guide to Observability … 6 The Hitchhiker's Guide to Observability … 7 The Hitchhiker's Guide to Observability … 8 The Hitchhiker's Guide to Observability …
Previous: The Hitchhiker's Guide to …

See Also:

  • The Hitchhiker's Guide to Observability - Here Comes Grafana - Part 7 –

    While we have been using the integrated tracing UI in OpenShift, it is time to summon Grafana. Grafana is a visualization powerhouse that allows teams to build custom dashboards, correlate traces with logs and metrics, and gain deep insights into their applications. In this article, we’ll deploy a dedicated Grafana instance for team-a in their namespace, configure a Tempo datasource, and create a dashboard to explore distributed traces.

  • The Hitchhiker's Guide to Observability Introduction - Part 1 –

    With this article I would like to summarize and, especially, remember my setup. This is Part 1 of a series of articles that I split up so it is easier to read and understand and not too long. Initially, there will be 6 parts, but I will add more as needed.

  • The Hitchhiker's Guide to Observability - Grafana Tempo - Part 2 –

    After covering the fundamentals and architecture in Part 1, it’s time to get our hands dirty! This article walks through the complete implementation of a distributed tracing infrastructure on OpenShift. We’ll deploy and configure the Tempo Operator and a multi-tenant TempoStack instance. For S3 storage we will use the integrated OpenShift Data Foundation. However, you can use whatever S3-compatible storage you have available.

  • The Hitchhiker's Guide to Observability - Central Collector - Part 3 –

    With the architecture defined in Part 1 and TempoStack deployed in Part 2, it’s time to tackle the heart of our distributed tracing system: the Central OpenTelemetry Collector. This is the critical component that sits between your application namespaces and TempoStack, orchestrating trace flow, metadata enrichment, and tenant routing.

  • The Hitchhiker's Guide to Observability - Example Applications - Part 4 –

    With the architecture defined, TempoStack deployed, and the Central Collector configured, we’re now ready to complete the distributed tracing pipeline. It’s time to deploy real applications and see traces flowing through the entire system! In this fourth installment, we’ll focus on the application layer - deploying Local OpenTelemetry Collectors in team namespaces and configuring example applications to generate traces. You’ll see how applications automatically get enriched with Kubernetes metadata, how namespace-based routing directs traces to the correct TempoStack tenants, and how the entire two-tier architecture comes together.

Copyright © 2020 - 2025 Toni Schmidbauer & Thomas Jungbauer