Creating an auditd rule to monitor Ansible Automation Platform static secrets

Tue, 09 Jun 2026 00:00:00 +0000

Ansible Automation Platform (AAP) comes with an elaborate credential management solution. All credentials within AAP are stored encrypted in a PostgreSQL database. But the keys used for encrypting database fields are currently stored on the machine(s) running AAP. In this blog post, we show we created an auditd rule to get at least some kind of monitoring if those keys are accessed.

Ansible Automation Platform local SECRET_KEY

AAP comes with a credential management solution. Credentials are used to

access remote machines
provide configuration data for automation, like API keys
connect AAP to source code repositories and inventories

and much more. You can even integrate external credential management solutions like HashiCorp Vault or OpenBao.

All credentials stored within AAP are saved as encrypted database fields in a PostgreSQL database. There is a main key and AAP creates for every encrypted database field a derived AES-256 encryption key. For more details see the documentation.

But there is one caveat: the main secret key is stored on the local file system. There is currently no supported option to integrate something like a HSM (High Security Module) for storing the main secret securely.

To mitigate this issue we deployed a Linux audit rule that monitors read/write and attribute changes to the main secret.

Configuring Linux auditd

We use the containerized version of AAP 2.6. In the containerized version of AAP those secrets are stored as podman secrets as the user running the AAP containers.

To list AAP podman secrets execute

$ podman secret list
ID NAME DRIVER CREATED UPDATED
02a2eb397dc5cedefde6ae411 hub_resource_server file 2 hours ago 2 hours ago
0dd84dc7b808917c4efb6ea9f controller_channels file 2 hours ago 2 hours ago
85fb73ff57729250c019976ed hub_database_fields file 2 hours ago 2 hours ago
c482ff7c07f908e443d327293 controller_postgres file 2 hours ago 2 hours ago
e6750121a8997cce8185072bf gateway_admin_password file 2 hours ago 2 hours ago
2731b2d03806f88f35809de66 eda_admin_password file 2 hours ago 2 hours ago
31ca44577b4c5b61dd3f0a0b6 eda_secret_key file 2 hours ago 2 hours ago
856473626bad9ff37b1b979d4 eda_resource_server file 2 hours ago 2 hours ago
b20ec0897fc2b6533544c4441 gateway_redis_url file 2 hours ago 2 hours ago
1dfdd812847ed4c1c01b194e2 gateway_db_password file 2 hours ago 2 hours ago
32d04575881dd781be192ef49 eda_db_password file 2 hours ago 2 hours ago
5102f69350717c15177b80821 hub_secret_key file 2 hours ago 2 hours ago
d1faf92922182604cfa746318 controller_resource_server file 2 hours ago 2 hours ago
d29f0d8c2fc709655dd918f2c postgresql_admin_password file 2 hours ago 2 hours ago
1b3ee24261bdff50d2c70e3f0 gateway_secret_key file 2 hours ago 2 hours ago
ae6c9e78e91dea180d22db6ee hub_settings file 2 hours ago 2 hours ago
b348d89bfb7d3b18f33a67d07 controller_secret_key file 2 hours ago 2 hours ago (1)

1	the controller secret key

If we want to know where those secrets are stored we can execute

$ podman secret inspect controller_secret_key
[
{
"ID": "b348d89bfb7d3b18f33a67d07",
"CreatedAt": "2026-06-09T10:57:28.278138715Z",
"UpdatedAt": "2026-06-09T10:57:28.278138715Z",
"Spec": {
"Name": "controller_secret_key",
"Driver": {
"Name": "file",
"Options": {
"path": "/home/admin/.local/share/containers/storage/secrets/filedriver"
}
},
"Labels": {}
}
}
]

Let’s see what’s in the filedriver directory:

$ ls /home/admin/.local/share/containers/storage/secrets/filedriver
secretsdata.json secretsdata.lock(1)

1	we are mainly interested in the secretsdata.json file

So the secretsdata.json file stores all podman secrets that are using the filedriver. For more information see the section Secret Drivers in the podman-secrets-create(1) manpage.

Let’s create a Linux auditd rule to monitor access to this file. As we are using Red Hat Enterprise Linux 10 a more detailed introduction to the Linux audit system is available here.

We created the file /etc/audit/rules.d/aap_secrets.rules with the following content:

-w /home/admin/.local/share/containers/storage/secrets/filedriver/secretsdata.json -p rwa -k aap_secrets (1)

1	-p rwa means monitor read/write/attribute changes, -k tells the audit system log events with an additional key "aap_secrets"

To activate the rule we need to load the audit rule via:

# auditctl -R /etc/audit/rules.d/aap_secrets.rules

You can list loaded audit rules via

# auditctl -l
-w /home/admin/.local/share/containers/storage/secrets/filedriver/secretsdata.json -p rwa -k aap_secrets

As a test we can cat the secrets file and check if auditd logged an event:

# cat /home/admin/.local/share/containers/storage/secrets/filedriver/secretsdata.json
...
# grep aap_secrets /var/log/audit/audit.log
type=SYSCALL msg=audit(1781009319.993:204): arch=c000003e syscall=257 success=yes exit=3 a0=ffffff9c a1=7ffc5276b65c a2=0 a3=0 items=1 ppid=6186 pid=6695 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts2 ses=3 comm="cat" exe="/usr/bin/cat" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="aap_secrets"ARCH=x86_64 SYSCALL=openat AUID="admin" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root" (1)

1	We can see that the cat command (comm=cat) was used by the root user to display the file contents

Summary

In this blog post we demonstrated how to configure the Linux audit system to log events if the AAP database secret key is accessed. The audit subsystem can also monitor system calls, for a detailed introduction see Audit system architecture.

Secrets on TechBlog about OpenShift/Ansible/Satellite and much more

Creating an auditd rule to monitor Ansible Automation Platform static secrets

Ansible Automation Platform local SECRET_KEY

Configuring Linux auditd

Summary