Ansible on TechBlog about OpenShift/Ansible/Satellite and much more

Creating an auditd rule to monitor Ansible Automation Platform static secrets

Tue, 09 Jun 2026 00:00:00 +0000

Ansible Automation Platform (AAP) comes with an elaborate credential management solution. All credentials within AAP are stored encrypted in a PostgreSQL database. But the keys used for encrypting database fields are currently stored on the machine(s) running AAP. In this blog post, we show we created an auditd rule to get at least some kind of monitoring if those keys are accessed.

Ansible Automation Platform local SECRET_KEY

AAP comes with a credential management solution. Credentials are used to

access remote machines
provide configuration data for automation, like API keys
connect AAP to source code repositories and inventories

and much more. You can even integrate external credential management solutions like HashiCorp Vault or OpenBao.

All credentials stored within AAP are saved as encrypted database fields in a PostgreSQL database. There is a main key and AAP creates for every encrypted database field a derived AES-256 encryption key. For more details see the documentation.

But there is one caveat: the main secret key is stored on the local file system. There is currently no supported option to integrate something like a HSM (High Security Module) for storing the main secret securely.

To mitigate this issue we deployed a Linux audit rule that monitors read/write and attribute changes to the main secret.

Configuring Linux auditd

We use the containerized version of AAP 2.6. In the containerized version of AAP those secrets are stored as podman secrets as the user running the AAP containers.

To list AAP podman secrets execute

$ podman secret list
ID NAME DRIVER CREATED UPDATED
02a2eb397dc5cedefde6ae411 hub_resource_server file 2 hours ago 2 hours ago
0dd84dc7b808917c4efb6ea9f controller_channels file 2 hours ago 2 hours ago
85fb73ff57729250c019976ed hub_database_fields file 2 hours ago 2 hours ago
c482ff7c07f908e443d327293 controller_postgres file 2 hours ago 2 hours ago
e6750121a8997cce8185072bf gateway_admin_password file 2 hours ago 2 hours ago
2731b2d03806f88f35809de66 eda_admin_password file 2 hours ago 2 hours ago
31ca44577b4c5b61dd3f0a0b6 eda_secret_key file 2 hours ago 2 hours ago
856473626bad9ff37b1b979d4 eda_resource_server file 2 hours ago 2 hours ago
b20ec0897fc2b6533544c4441 gateway_redis_url file 2 hours ago 2 hours ago
1dfdd812847ed4c1c01b194e2 gateway_db_password file 2 hours ago 2 hours ago
32d04575881dd781be192ef49 eda_db_password file 2 hours ago 2 hours ago
5102f69350717c15177b80821 hub_secret_key file 2 hours ago 2 hours ago
d1faf92922182604cfa746318 controller_resource_server file 2 hours ago 2 hours ago
d29f0d8c2fc709655dd918f2c postgresql_admin_password file 2 hours ago 2 hours ago
1b3ee24261bdff50d2c70e3f0 gateway_secret_key file 2 hours ago 2 hours ago
ae6c9e78e91dea180d22db6ee hub_settings file 2 hours ago 2 hours ago
b348d89bfb7d3b18f33a67d07 controller_secret_key file 2 hours ago 2 hours ago (1)

1	the controller secret key

If we want to know where those secrets are stored we can execute

$ podman secret inspect controller_secret_key
[
{
"ID": "b348d89bfb7d3b18f33a67d07",
"CreatedAt": "2026-06-09T10:57:28.278138715Z",
"UpdatedAt": "2026-06-09T10:57:28.278138715Z",
"Spec": {
"Name": "controller_secret_key",
"Driver": {
"Name": "file",
"Options": {
"path": "/home/admin/.local/share/containers/storage/secrets/filedriver"
}
},
"Labels": {}
}
}
]

Let’s see what’s in the filedriver directory:

$ ls /home/admin/.local/share/containers/storage/secrets/filedriver
secretsdata.json secretsdata.lock(1)

1	we are mainly interested in the secretsdata.json file

So the secretsdata.json file stores all podman secrets that are using the filedriver. For more information see the section Secret Drivers in the podman-secrets-create(1) manpage.

Let’s create a Linux auditd rule to monitor access to this file. As we are using Red Hat Enterprise Linux 10 a more detailed introduction to the Linux audit system is available here.

We created the file /etc/audit/rules.d/aap_secrets.rules with the following content:

-w /home/admin/.local/share/containers/storage/secrets/filedriver/secretsdata.json -p rwa -k aap_secrets (1)

1	-p rwa means monitor read/write/attribute changes, -k tells the audit system log events with an additional key "aap_secrets"

To activate the rule we need to load the audit rule via:

# auditctl -R /etc/audit/rules.d/aap_secrets.rules

You can list loaded audit rules via

# auditctl -l
-w /home/admin/.local/share/containers/storage/secrets/filedriver/secretsdata.json -p rwa -k aap_secrets

As a test we can cat the secrets file and check if auditd logged an event:

# cat /home/admin/.local/share/containers/storage/secrets/filedriver/secretsdata.json
...
# grep aap_secrets /var/log/audit/audit.log
type=SYSCALL msg=audit(1781009319.993:204): arch=c000003e syscall=257 success=yes exit=3 a0=ffffff9c a1=7ffc5276b65c a2=0 a3=0 items=1 ppid=6186 pid=6695 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts2 ses=3 comm="cat" exe="/usr/bin/cat" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="aap_secrets"ARCH=x86_64 SYSCALL=openat AUID="admin" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root" (1)

1	We can see that the cat command (comm=cat) was used by the root user to display the file contents

Summary

In this blog post we demonstrated how to configure the Linux audit system to log events if the AAP database secret key is accessed. The audit subsystem can also monitor system calls, for a detailed introduction see Audit system architecture.

Using a local Renovate bot to manage Ansible collection dependencies

Tue, 26 May 2026 00:00:00 +0000

We wanted to use renovate bot to automatically update collection dependencies for one of our Ansible playbook repositories. This blog post describes the basics of renovate, how we configured it and how we actually run renovate locally.

Understanding renovate

As this was our first try of using renovate for dependency updates, we struggled to understand how renovate works and how it is configured.

Renovate is a bot that is able to scan multiple repositories for dependencies and create pull/merge requests with updated dependencies.

Configuration is split between a configuration for the bot and a local one for each repository.

So you basically have

One or more global configuration files that configure the bot (config.js) written in JavaScript or TypeScript
One configuration file per repository (renovate.json) written in JSON that contains local configuration options just for this repository.

Renovate is very flexible on how to structure the configuration. For more information see config-overview in the docs.

Renovate also supports multiple platforms, like GitHub, GitLab or Forgejo. For a complete list see renovate platforms.

The idea is that you run the bot once, it scans all configured repositories and creates pull requests in every repository with dependency updates.

If your projects are hosted on GitHub and you would like to use GitHub Actions for running renovate the straight forward solution is to follow Mend Renovate on GitHub Marketplace.

But we wanted more control and tried to

run renovate on a local machine. This should be migrated to a Job Template or Kubernetes CronJob later (might be worth another post)
use a private Automation Hub for hosting dependencies. So renovate should query collections on our Private Automation Hub for dependency updates
credentials should be made available via environment variables to the bot

Configuring and running renovate

As a first test we created a local bot configuration file config.js in our home directory. The GIT repository we used for testing is https://github.com/tosmi-ansible/aap-setup.

The first thing to be aware of is that the renovate bot by default always searches the repository configuration file (renovate.json) in the upstream repository. Changes to local files are ignored.

One way to avoid this is using --platform=local. This actually uses the local renovate.json file, but has the caveat that not all renovate functionality is available.

But in our case this was good enough as we just wanted to test AAP Hub authentication.

It is also helpful to run renovate with debugging enabled, this dumps the configuration currently used to STDOUT.

For example we had the following renovate.json config for testing:

{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"config:best-practices"
],
"timezone": "Europe/London",
"schedule": [
"at any time"
],
"packageRules": [
{
"matchDatasources": ["galaxy-collection"]
},
{
"matchDatasources": ["github-tags"],
"enabled": "false"
}
],
"hostRules": [
{
"matchHost": "aap.apps.hub.aws.tntinfra.net",
"token": "Token <the token>",
"authType": "Token-Only"
}
]
}

But running renovate with LOG_LEVEL=debug set in the environment revealed the following settings:

DEBUG: Repository config (repository=tosmi-ansible/aap-setup)
"fileName": "renovate.json",
"config": {
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:recommended"]
}

The configuration above is the configuration available in the remote GIT repository and not our local configuration.

Renovate does not use a local renovate.json repository config by default. It always pulls the configuration from the remote repository except when you use _—platform=local.

Furthermore renovate heavily relies on local caching. We wanted to test if renovate is using our hub token for connecting to the Hub, but we could not see any log entry in the AAP Hub container.

Renovate creates its cache files in a directory called baseDir. You can specify the location (and later wipe the cache) with the --base-dir option.

As we are using a self signed certificate in our AAP Hub test installation we had to specify NODE_TLS_REJECT_UNAUTHORIZED=0 to disable TLS verification.

To summarize our test configuration:

We used the following config.js global renovate configuration file:

module.exports = {
token: '<github token>',
platform: 'github',
onboardingConfig: {
extends: ['config:recommended'],
},
//repositories: ['tosmi-ansible/aap-setup'], (1)
};

1	Repository listing is not allowed when using --platform=local

In the repository where we want to test automatic dependency updates we used the following renovate.json configuration:

{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"config:best-practices"
],
"timezone": "Europe/London",
"schedule": [
"at any time"
],
"packageRules": [
{
"matchDatasources": ["galaxy-collection"]
},
{
"matchDatasources": ["github-tags"],
"enabled": "false" (1)
}
],
"hostRules": [
{
"matchHost": "aap.apps.hub.aws.tntinfra.net",
"token": "Token <the token>",
"authType": "Token-Only"
}
]
}

1	We disabled GitHub tag lookup explicitly as renovate would find some of our dependencies on GitHub

This is our collections/requirements.yaml file with a test dependency:

roles:
collections:
- name: kubernetes.core
version: 6.3.0
source: https://aap.apps.hub.aws.tntinfra.net/api/galaxy/content/rh-certified/

On our AAP Private Automation Hub we had kubernetes.core version 6.4.0 available.

So to run and test renovate locally we used the following command line:

NODE_TLS_REJECT_UNAUTHORIZED=0 LOG_LEVEL=debug RENOVATE_CONFIG_FILE=~/config.js renovate --platform=github --repository-cache=disabled --base-dir=/tmp/cache

Secret management and configuration cleanup

The next step is to remove the hard-coded tokens from our configuration. We wanted to use environment variables, as this allows us to run renovate as:

A Kubernetes CronJob with a Pod. We could inject the token via a Secret
As a job template within AAP with tokens stored as AAP credentials or
A normal Unix cron job with environment variables set

As the global renovate configuration config.js is written in JavaScript we could leverage process.env to retrieve required tokens from environment variables.

Here is our modified config.js:

module.exports = {
platform: 'github',
onboardingConfig: {
extends: ['config:recommended'],
},
// repositories: ['tosmi-ansible/aap-setup'], (1)
secrets: {
AAP_HUB_TOKEN: process.env.AAP_HUB_TOKEN, (2)
},
hostRules: [
{
matchHost: "aap.apps.hub.aws.tntinfra.net",
token: "{{ secrets.AAP_HUB_TOKEN }}", (3)
authType: "Token" (4)
},
],
};

1	We disable repository listing for --platform=local. Otherwise renovate will raise an error.
2	We define a new secret AAP_HUB_TOKEN and read the value from the environment variable AAP_HUB_TOKEN
3	We use the secret as our token for authenticating to AAP Hub
4	The authType setting is a little bit special, see the following section

For GitHub authentication renovate expects a GitHub token in the environment variable RENOVATE_TOKEN.

A few words about the authType setting. AAP Hub requires the following authorization header in the HTTP request for authentication:

Authorization: Token <the token>

With renovate we have two options to achieve this:

Either set authType to "Token-Only" and specify the token as "Token <the token>". So with token-only renovate takes the token string verbatim
Or set the authType to the string we would like to prepend to the token as we did

We had to dig into the source code of renovate to understand how it functions.

And here is the renovate.json repository configuration that we used:

{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"config:best-practices"
],
"timezone": "Europe/London",
"schedule": [
"at any time"
],
"packageRules": [
{
"matchDatasources": ["galaxy-collection"]
},
{
"matchDatasources": ["github-tags"],
"enabled": "false"
}
]
}

You could also move the hostRules to the repository local renovate.json file and still use secrets.AAP_HUB_TOKEN, but as this is going to be our global configuration for Ansible code base covering multiple repositories our current idea is to store the authentication in the global configuration.

To set the required environment variables containing our secrets we execute

$ export AAP_HUB_TOKEN=$(pass show other/ansible/aws-aap-tokens | awk '/hub/ {print $2}') (1)
$ export RENOVATE_TOKEN=$(pass show other/github/tosmi-rw-token | awk '/token/ {print $2}')

1	We use the pass utility for secret management.

Integration with GitHub

So the last step is to store everything (the global and local configuration) on GitHub.

For the global repository we created the tosmi-ansible/renovate-config repository.

Here is our final config.js global configuration file in this repository:

module.exports = {
platform: 'github',
gitAuthor: "Renovate Bot <renovate-bot@stderr.at>",
dryRun: null,
onboardingConfig: {
extends: ['config:recommended'],
},
repositories: ['tosmi-ansible/aap-setup'], (1)
secrets: {
AAP_HUB_TOKEN: process.env.AAP_HUB_TOKEN,
},
hostRules: [
{
matchHost: "aap.apps.hub.aws.tntinfra.net",
token: "{{ secrets.AAP_HUB_TOKEN }}",
authType: "Token"
},
],
};

1	This time we tell renovate which repositories it should scan.

The default configuration default.json, also stored in the global repository:

{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"config:best-practices"
],
"timezone": "Europe/London",
"schedule": [
"at any time"
],
"enabledManagers": ["ansible", "ansible-galaxy"], (1)
"packageRules": [
{
"matchDatasources": ["galaxy-collection"]
},
{
"matchDatasources": ["github-tags"],
"enabled": "false"
}
]
}

1	We only enable ansible and ansible-galaxy managers. Otherwise renovate would search for various other sources for dependencies.

tosmi-ansible/aap-setup contains our local configuration:

{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"tosmi-ansible/renovate-config",
"config:best-practices"
]
}

This only contains a reference to our global configuration and we would like to use the best-practices preset.

We can now run renovate from the renovate-config repository and it will scan the configured repository and create a pull request with updated dependencies if required.

$ git clone git@github.com:tosmi-ansible/renovate-config.git
$ cd renovate-config
$ NODE_TLS_REJECT_UNAUTHORIZED=0 renovate --platform=github --base-dir=/tmp/renovate-cache (1)

1	We still need to set NODE_TLS_REJECT_UNAUTHORIZED to 0, because our AAP Hub instance uses a private certificate

One minor issue that we discovered is that if a pull request for an update already existed (even a closed one) Renovate would not open a new one. If you run renovate with LOG_LEVEL=debug you will see the following message:

DEBUG: Closed PR #19 already exists. Skipping branch. (repository=tosmi-ansible/aap-setup, branch=renovate/kubernetes.core-6.x)
"prTitle": "Update dependency kubernetes.core to v6.4.0"

To convince Renovate to create a new pull request just rename the closed pull request on GitHub. We just added the prefix "RENAMED". After renaming the pull request, renovate will happily create a new one.

Tips and tricks

This section list various things we learning while running renovate locally

Renovate does not honor new dependencies in requirements.yaml

We had an issues with new dependencies in requirements.yaml. Wiping the renovate cache (/tmp/renovate_cache) in our case fixed this and renovate created new pull requests for the added dependencies.

Renovate does not open new PR (pull requests) for updated dependencies

We double checked AAP and our renovate configuration and there was clearly a newer version of a collection but renovate just did not open a new PR.

Running renovate with LOG_LEVEL=debug revealed the following message (we tried to update the redhat.openshift_virtualization collection):

 {
"branchName": "renovate/redhat.openshift_virtualization-2.x",
"prNo": null,
"prTitle": "Update dependency redhat.openshift_virtualization to v2.2.4",
"result": "branch-limit-reached", (1)

1	renovate does not open a new branch because of "branch-limit-reached"

We check our upstream repository on GitHub and there was just the main branch, hm…

Looking further up in the log we found:

DEBUG: prHourlyLimit of the upgrades present in this branch (repository=tosmi-ansible/aap-setup, branch=renovate/infra.aap_utilities-3.x)
"limits": [{"depName": "infra.aap_utilities", "prHourlyLimit": 2}]
DEBUG: Calculated lowest prHourlyLimit among the upgrades present in this branch is 2. (repository=tosmi-ansible/aap-setup, branch=renovate/infra.aap_utilities-3.x)
DEBUG: Hourly PRs limit reached (repository=tosmi-ansible/aap-setup, branch=renovate/infra.aap_utilities-3.x) (1)
"hourlyPrCount": 2,
"hourlyPrLimit": 2
DEBUG: Reached branch limit - skipping branch creation (repository=tosmi-ansible/aap-setup, branch=renovate/infra.aap_utilities-3.x)

1	There is a hourly branch limit of two in place

As we just ran renovate to update another dependency, we seem to be hitting this hourly limit. The docs mention a default of 2.

The solution was to run renovate with the option --pr-hourly-limit=0:

NODE_TLS_REJECT_UNAUTHORIZED=0 renovate --platform=github --base-dir=/tmp/renovate-cache --pr-hourly-limit=0

Onboarding to Ansible Automation Platform with Configuration as Code

Wed, 04 Mar 2026 00:00:00 +0000

We had the honor of presenting at the 2nd Ansible Anwendertreffen in Austria. The topic was the onboarding of application teams to the Ansible Automation Platform.

We created an extensive demo that:

Onboards a new tenant into an AAP organization.
Each tenant gets a configuration as code repository, cloned from a template.
A push event to the config-as-code repository triggers an update of AAP objects.
Provides an example repository for each tenant to get started.
The example repository provides a webhook that creates an OpenShift Virtualization VM for testing code changes when a feature branch is created.
Contains a playbook to display objects that are not currently managed by the tenant’s configuration-as-code repository.

The source code for the demo is available on GitHub.

The README contains more details about the implementation, and the slide deck is also available on GitHub.

Ansible Style Guide

Tue, 30 Nov 2021 00:00:00 +0000

You should always follow the Best Practices and Ansible Lint rules defined by the Ansible documentation when developing playbooks.

Although very basic, the Best Practices document gives a few guidelines to be able to carry out well-structured playbooks and roles, it contains recommendations that evolve with the project, so it is recommended to review it regularly. It is advisable to review the organization of content in Ansible.

The Ansible Lint documentation shows us through this tool the syntax rules that will be checked in the testing of roles and playbooks, the rules that will be checked are indicated in this document in their respective section.

This style guide is meant as a base of playbook development. It is not the ultimate truth. Always verify and apply appropriate company rules. There are many additional references available, such as: https://github.com/whitecloud/ansible-styleguide or https://github.com/redhat-cop/automation-good-practices

General Recommendations

Treat your Ansible content like code: Any playbook, role, inventory or collection should be versionized using Git or a similar tool.
Iterate: start with basic playbook and refactor later
Use multiple Git repositories: on per role/collection, separate inventory from other repositories
Use human-meaningful names in inventory files
group hosts in inventory files
use a consistent Directory Structure

Optimize Playbook Execution

Disable facts gathering if it is not required.
1. Try not to use: ansible_facts[‘hostname’] (or ‘nodename’)
2. Try to use inventory_hostname and inventory_hostname_short instead
3. It is possible to selectively gather facts (gather_subnet)
4. Consider caching facts
Increase Parallelism
1. Ansible runs 1st task on every host, then 2nd task on every host …
2. Use “forks” (default is 5) to control how many connections can be active (load will increase, try first with conservative values)
3. While testing forks analize Enable CPU and Memory Profiling
If you use the module "copy" for large files: do not use “copy” module. Use “synchronize” module instead (based on rsync)
Use lists when ever a modules support it (i.e. yum) Instead of

tasks:
- name: Ensure the packages are installed
yum:
name: "{{ item }}"
state: present
loop:
- httpd
- mod_ssl
- httpd-tools
- mariadb-server
- mariadb
- php
- php-mysqlnd

use

tasks:
- name: Ensure the packages are installed
yum:
state: present
name:
- httpd
- mod_ssl
- httpd-tools
- mariadb-server
- mariadb
- php
- php-mysqlnd

Optimize SSH Connections
1. in ansible.cfg set
  1. ControlMaster
  2. ControlPersist
  3. PreferredAuthentications
Enabling Pipelining
1. Not enabled by default
2. reduces number of SSH operations
3. requires to disable requiertty in sudo options

Name Tasks and Plays

Every task or play should be explicitly named in a way that the purpose is easily understood and can be referred to if the playbook has to be restarted from a certain task.

For example the following is hard to read and debug:

PLAY [localhost]
********************************
TASK [include_vars]
********************************
ok: [localhost]
TASK [yum]
********************************
ok: [localhost]

While with naming it is easier to follow what is happening:

PLAY [Create a new virtual machine]
********************************
TASK [Include vmware-credentials]
********************************
ok: [localhost]
TASK [Install required packages with yum]
********************************
ok: [localhost]

# bad
# set another variable
- set_fact:
my_second_var: "{{ my_var }}"

# good
- name: set another variable
set_fact:
my_second_var: "{{ my_var }}"

Reason
Better understanding what is currently happening in a play and better possibility to debug.

Variables in Task Names

Include as much information as necessary to explain the purpose of a task. Make usage of variables inside a task name to create dynamic output messages.

#bad
- name: 'Change status'
service:
enabled: true
name: 'httpd'
state: '{{ state }}'
become: true

#good
- name: 'Change status of httpd to {{ state }}'
service:
enabled: true
name: 'httpd'
state: '{{ state }}'
become: true

Reason
This will help to easily understand log outputs of playbooks.

Omitting Unnecessary Information

While name tasks in a playbook, do not include the name of the role which is currently executed, since Ansible will do this automatically.

Reason
Avoiding the same output twice on the console will prevent confusions.

Names

All the newly created Ansible roles should follow the name convention using dashes if necessary: [company]-[action]-[function/technology]

# bad
lvm

# good
mycompany-setup-lvm

Reason
If using roles from Ansible Galaxy, it will keep consistency about which roles are created internally.

Use Modules instead of command or shell

Before using the command or shell module, verify if there is already a module available which can avoid the usage of raw shell command.

# bad
- name: install httpd
tasks:
- command: "yum install httpd"

# good
- name: install packages
tasks:
- name: 'install httpd'
yum:
name: 'httpd'
state: 'present'

Reason
While raw command could be seen as a security risk in general, another reason to avoid them is the loss of immutability of the ansible playbooks or roles. Ansible cannot verify if a command has been already executed before or not and will therefore execute it every time the playbook is running.

Documenting a Task/Play

Every playbook, role or task should start with a documentation why this code has been written and what it does and should include an example usage if applicable. The comment should be followed by ---` with no blank lines around it, to indicate the actual start of the yaml definition

#bad
- name: 'Change httpd status'
service:
enabled: true
name: 'httpd'
state: '{{ state }}'
become: true

#good
# Example usage: ansible-playbook -e state=started playbook.yml
# This playbook changes the state of the httpd daemon
---
- name: 'Change httpd status'
service:
enabled: true
name: 'httpd'
state: '{{ state }}'
become: true

Reason
This common programmatic practice helps to quickly understand the purpose and usage of a playbook or role.

Lint rule
Yamllint document-start rule

End a File

Files should be ended with a newline.

Reason
This is common Unix best practice which avoids any prompt misalignment when printing files in a terminal.

Lint rule
Yamllint new-line-at-end-of-file rule

Quotes

Strings should be quoted, while quotes for booleans (e.g. true/false) or integers (i.g. 42) should be avoided.

Do NOT quote:

hosts: targets (e.g. hosts: databases rather than hosts: ‘databases’)
include_tasks: and include_roles: target file names
task and role names
registered variables
number values
boolean values

It is possible to use single or double quotes. In this document single quotes are used, as it seems to be more common. Double quotes are often seen in European countries, especially German speaking countries. The most important thing is to stick to one style.

# bad
- name: 'start robot named my-robot'
service:
name: my-robot
state: started
enabled: true
become: true

# good
- name: 'start robot named my-robot'
service:
name: 'my-robot'
state: 'started'
enabled: true
become: true
# double quotes w/ nested single quotes
- name: 'start all robots'
service:
name: '{{ item["robot_name"] }}'
state: 'started'
enabled: true
with_items: '{{ robots }}'
become: true
# double quotes to escape characters
- name 'print some text on two lines'
debug:
msg: "This text is on\ntwo lines"
# folded scalar style
- name: 'robot infos'
debug:
msg: >
Robot {{ item['robot_name'] }} is {{ item['status'] }} and in {{ item['az'] }}
availability zone with a {{ item['curiosity_quotient'] }} curiosity quotient.
with_items: robots
# folded scalar when the string has nested quotes already
- name: 'print some text'
debug:
msg: >
"I haven't the slightest idea," said the Hatter.
# do not quote booleans/numbers
- name: 'download google homepage'
get_url:
dest: '/tmp'
timeout: 60
url: 'https://google.com'
validate_certs: true
# variables example 1
- name: 'set a variable'
set_fact:
my_var: 'test'
# variables example 2
- name: 'print my_var'
debug:
var: my_var
when: ansible_os_family == 'Darwin'
# variables example 3
- name: 'set another variable'
set_fact:
my_second_var: '{{ my_var }}'

Lint rule
Yamllint quoted-strings rule

Sudo

Use the new become syntax when designating that a task needs to be run with sudo privileges

#bad
- name: 'template client.json to /etc/sensu/conf.d/'
template:
dest: '/etc/sensu/conf.d/client.json'
src: 'client.json.j2'
sudo: true

# good
- name: 'template client.json to /etc/sensu/conf.d/'
template:
dest: '/etc/sensu/conf.d/client.json'
src: 'client.json.j2'
become: true

Reason
Using sudo was deprecated at Ansible version 1.9.1

Lint rule
Ansible-lint E103 rule

Hosts Declarations

Host sections be defined in such a way that it follows this general order:

host declaration
host options in alphabetical order
pre_tasks
roles
tasks

# example
- hosts: 'webservers'
remote_user: 'centos'
vars:
tomcat_state: 'started'
pre_tasks:
- name: 'set the timezone to America/Boise'
lineinfile:
dest: '/etc/environment'
line: 'TZ=America/Boise'
state: 'present'
become: true
roles:
- { role: 'tomcat', tags: 'tomcat' }
tasks:
- name: 'start the tomcat service'
service:
name: 'tomcat'
state: '{{ tomcat_state }}'

Reason
A global definition about the order of these items, will create an easy and consistent human readable code.

Tasks Declarations

A task should be defined in such a way that it follows this general order:

task name
tags
task map declaration (e.g. service:)
task parameters in alphabetical order (remember to always use multi-line map syntax)
loop operators (e.g. with_items)
task options in alphabetical order (e.g. become, ignore_errors, register)

# example
- name: 'create some ec2 instances'
tags: 'ec2'
ec2:
assign_public_ip: true
image: 'ami-c7d092f7'
instance_tags:
Name: '{{ item }}'
key_name: 'my_key'
with_items: '{{ instance_names }}'
ignore_errors: true
register: ec2_output
when: ansible_os_family == 'Darwin'

Reason
A global definition about the order of these items, will create an easy and consistent human readable code.

Include Declaration

For include statements, make sure to quote filenames and only use blank lines between include statements if they are multi-line (e.g. they have tags).

# bad
- include: other_file.yml
- include: 'second_file.yml'
- include: third_file.yml tags=third

# good
- include: 'other_file.yml'
- include: 'second_file.yml'
- include: 'third_file.yml'
tags: 'third'

Reason
Using such syntax, will create an easy and consistent human readable code.

Booleans

Use true/false instead of yes/no (or 1/0).

# bad
- name: 'start httpd'
service:
name: 'httpd'
state: 'restarted'
enabled: 1
become: 'yes'

# good
- name: 'start httpd'
service:
name: 'httpd'
state: 'restarted'
enabled: true
become: true

Reason
Ansible can read boolean values in many different ways, like: True/False, true/false, yes/no, 1/0. It makes sense to stick to one option, which is then equally used in any playbook or role. It is recommended to use true/false since Java and Javascript are using the same values for boolean values.

Lint rule
Yamllint truthy rule

Required config: truthy: {allowed-values: ["true", "false"]}`

Key Value Pairs

Only one space should be used after the colon, when defining key/value pairs.

# bad
- name : 'start httpd'
service:
name : 'httpd'
state : 'restarted'
enabled : true
become : true

# good
- name: 'start httpd'
service:
name: 'httpd'
state: 'restarted'
enabled: true
become: true

Reason
It increases human readability and reduces changeset collisions for version control.

Lint rule
Yamllint colons rule

Required config: colons: {max-spaces-before: 0, max-spaces-after: 1}

Using Map Syntax

Always use the map syntax for better readability.

# bad
- name: 'create conf.d directory'
file: 'path=/etc/httpd/conf.d/ state=directory mode=0755 owner=httpd group=httpd'
become: true
- name: 'copy mod_ssl.conf to /etc/httpd/conf.d'
copy: 'dest=/etc/httpd/conf.d/ src=mod_ssl.conf'
become: true

# good
- name: 'create conf.d directory'
file:
group: 'httpd'
mode: '0755'
owner: 'httpd'
path: '/etc/httpd/conf.d'
state: 'directory'
become: true
- name: 'copy mod_ssl.conf to /etc/httpd/conf.d'
copy:
dest: '/etc/httpd/conf.d/'
src: 'mod_ssl.conf'
become: true

Reason
It increases human readability and reduces changeset collisions for version control.

Spacing

You should have blank lines between two host blocks, between two task blocks, and between host and include blocks. When indenting, you should use 2 spaces to represent sub-maps, and multi-line maps should start with a -. For a more in-depth example of how spacing (and other things) please take a look at the example below

# Example: ansible-playbook --ask-become-pass --ask-vault-pass style.yml
#
# This is a sample Ansible script to showcase all of our style decisions.
# Pay close attention to things like spacing, where we use quotes, etc.
# The only thing you can ignore is where comments are, except this first comment:
# It's generally a good idea to include some good information like sample usage
# at the beginning of your file, so that someone can run head on the script
# to see what they should do.
#
# A good rule of thumb on quoting is to quote anything that represents a value
# that does not represent either a primitive type, or something within the
# playbook; e.g. do not quote integers, booleans, variable names, boolean logic
# Variable names still need to be quoted when they are module parameters for
# Ansible to properly resolve them.
# You should also always have single quotes around the outer string, and
# double quotes on the inside.
# If for some reason this is not possible or it would require escaping quotes
# (which you should avoid if you can), use the scalar string operator (shown
# in this playbook).
#
# Directory structure style:
# Your directory structure should match the structure described by the Ansible
# developers: http://docs.ansible.com/ansible/playbooks_best_practices.html
#
# ---
#
# - include: 'role_name.yml'
# become: true # only if every task in the role requires super user
#
# The self-named yml file contains all of the actual role tasks.
#
# Header comments are followed by blank line, then --- to signify start of YAML,
# then another blank line, then the script.
---
- hosts: 'localhost'
tasks:
- name: 'fail if someone tries to run this'
fail:
msg: 'this playbook was not meant to actually be ran. just inspect the source!'
- include: 'first_include.yml' # quote filenames
- include: 'second_include.yml' # no blank line needed between includes without tags
- include: 'third_include.yml' # includes with tags should have blank lines between
tags: 'third_include'
- include: 'fourth_include.yml'
tags: 'fourth_include'
- hosts: 'tag_environment_samplefruit'
remote_user: 'centos' # options in alphabetical order
vars:
sample_str: 'dood' # use snake_case for variable names
sample_bool: true # do not quote booleans or integers
sample_int: 42
vars_files:
- 'group_vars/secrets.yml'
pre_tasks: # then pre_tasks, roles, tasks
- name: 'this runs a command that involves both single and double quotes'
command: >
echo "I can't even"
args:
chdir: '/tmp'
- name: 'this command just involves double quotes'
command: 'echo "Hey man"'
roles:
- { role: 'sample_role', tags: 'sample_role' } # use this format for role listing
tasks:
- name: 'get list of directory permissions in /tmp'
command: 'ls -l /tmp'
register: tmp_listing # do not quote variable names when registering
# A task should be defined in the following order:
# name
# tags
# module
# module arguments, alphabetical
# loop operator (e.g. with_items, with_fileglob)
# other options, alphabetical (e.g. become, ignore_errors, when)
- name: 'a more complicated task to show where everything goes: touch all items from /tmp'
tags: 'debug' # tags go immediately after name
file:
path: '{{ item }}' # use path for single file actions, dest/src for multi file actions
state: 'touch' # arguments go in alphabetical order
with_items: tmp_listing.stdout_lines # loop things go immediately after module
# the rest of the task options are in alphabetical order
become: true # try to keep become only on the tasks that need it. If every task in a host uses become, then move it up to the host options
ignore_errors: true
when: ansible_os_family == 'Darwin' and tmp_listing.stdout_lines | length > 1
- name: 'some modules can have maps in their maps (woah man)'
ec2:
assign_public_ip: true
group: ['wca_ssh', 'wca_tomcat']
image: 'ami-c7d092f7'
instance_tags:
Name: 'instance'
service_tomcat: ''
key_name: 'ops'
- hosts: 'tag_environment_secondfruit'
tasks:
- name: 'this task has multiple tags'
tags: ['tagme', 'tagmetoo']
set_fact:
mr_fact: 'w'
- name: 'perform an action'
action: ec2_facts
delegate_to: 'localhost'
# newline at end of file

Reason
This produces nice looking code that is easy to read.

Lint rule
Yamllint empty-lines rule

Variable Names

Names of variables should be as expressive as possible. snake_case for names will help to make the code human readable. The prefix should contain the name of the role.

# bad
- name: 'set some facts'
set_fact:
myBoolean: true
int: 20
MY_STRING: 'test'

# good
- name: 'set some facts'
set_fact:
rolename_my_boolean: true
rolename_my_int: 20
rolename_my_string: 'test'

Reason
Ansible uses snake_case for module names so it makes sense to extend this convention to variable names. Perfixing the variable with the role name, makes it immediately obvious where it is used.

Jinja Variables

Use spaces around Jinja variable names to increase readability.

# bad
- name: set some facts
set_fact:
my_new_var: "{{my_old_var}}"

# good
- name: set some facts
set_fact:
my_new_var: "{{ my_old_var }}"

Reason
A proper definition for how to create Jinja variables produces consistent and easily readable code.

Lint rule
Ansible-lint E206 rule

Comparing

Do not compare to literal True/False.
Use when: var rather than when: var == True (or conversely when: not var).

Do not compare it to empty strings.
Use when: var rather than when: var != "" (or conversely when: not var rather than when: var == "")

# bad
- name: validate required variables
fail:
msg: "No value specified for '{{ item }}'"
when: (vars[item] is undefined) or (vars[item] is defined and vars[item] | trim == "")
with_items: "{{ appd_required_variables }}"
- name: Create an user and add to the global group
include_tasks: user.yml
when:
- username is defined
- username != ""

# good
- name: Validate required variables
fail:
msg: "No value specified for '{{ item }}'"
when: (vars[item] is undefined) or (vars[item] is defined and not vars[item] | trim == "")
with_items: "{{ appd_required_variables }}"
- name: Create an user and add to the global group
include_tasks: user.yml
when:
- username is defined
- username

Reason
Avoid code complexity using quotes and standardize the way literals and empty strings are used

Lint rule
Ansible-lint E601 rule

Delegation

Do not use local_action, use delegate_to: localhost

# bad
- name: Send summary mail
local_action:
module: mail
subject: "Summary Mail"
to: "{{ mail_recipient }}"
body: "{{ mail_body }}"
run_once: true

# good
- name: Send summary mail
mail:
subject: "Summary Mail"
to: "{{ mail_recipient }}"
body: "{{ mail_body }}"
delegate_to: localhost
run_once: true

Reason
Avoid complexity, standardization, flexibility and code readability. The module and its parameters are easy to read and can be delegated even to a third party server.

Lint rule
Ansible-lint E504 rule

Playbook File Extension

All Ansible Yaml files should have a .yml extension (and NOT .YML, .yaml etc).

# bad
~/tasks.yaml

# good
~/tasks.yml

Reason
Ansible tooling (like ansible-galaxy init) creates files with a .yml extension. Also, the Ansible documentation website references files with a .yml extension several times. Because of this, it is normal in the Ansible community to use a .yml extension for all Ansible YAML files.

Lint rule
Ansible-lint E205 rule

Template File Extension

All Ansible Template files should have a .j2 extension.

# bad
~/template.conf

# good
~/template.conf.j2

Reason
Ansible Template files will usually have the .j2 extension, which denotes the Jinja2 templating engine used.

Vaults

All Ansible Vault files should have a .vault extension (and NOT .yml, .YML, .yaml etc).

# bad
~/secrets.yml

# good
~/secrets.vault

Reason
It is easier to control unencrypted files automatically for the specific .vault extension.

Debug and Comments

Do not overuse debug and comments in final code as much as possible. Use task and role names to explain what the task or role does. Use the verbose option under ansible for debugging purposes. If debug is used, assign a verbosity option. This will display the message only on certain debugging levels.

# bad
- name: print my_var
debug:
var: my_var
when: ansible_os_family == "Darwin"

# good
- name: print my_var
debug:
var: my_var
verbosity: 2
when: ansible_os_family == "Darwin"

Reason
It will keep clean code and consistency avoiding extra debug and comments. Extra debug will spend extra time when running the playbook or role.

Use Modules copy or templates instead of linefile or blockfile

Instead of using the modules linefile and blockfile, which manage changes inside a file, it should be tried to use the modules copy or template instead, which will manage the whole file. For better future proof template should be preferred over copy.

Reason
When using linefile/blockfile only a single line or a part of a file is managed. It is not easy to remember which part exactly is managed and which is not. Using the template module the whole file is managed by Ansible and there is no confusion about different parts of a file. Moreover, regular expressions can be avoided, which are often used using linefile.

Use Module synchronize Instead of copy for Large Files

Copying large files takes significantly longer than syncing it. The synchronize modules which are based on rsync can increase the time moving large files from one node to another (or even on the same node).

Do not Show Sensitive Data in Ansible Output

When using the template module and there are passwords or other sensitive data in the file, use the no_log option to hide this information.

Reason
For obvious reasons the output of sensitive data on the screen (and logfile) should be prohibited.

Use Block-Module

Block can help to organize the code and can enable rollbacks.

- block:
copy:
src: critical.conf
dest: /etc/critical/crit.conf
service:
name: critical
state: restarted
rescue:
command: shutdown -h now

Reason
Using blocks groups critical tasks together and allows better management when a single task of this block fails.

Enable CPU and Memory Profiling

Enabling profiling is extremely useful when testing forks or to analyse memory and cpu consumption in general. It will present a summary of used CPU/memory for the whole play and per task.

To enable it:

Create a new control group - which is required for CPU and memory profiling

cgcreate -a root:root -t root:root -g cpuacct,memory,pids:ansible_profile

Configure ansible.cfg

callback_whitelist=cgroup_perf_recap
[callback_cgroup_perf_recap]
control_group=ansible_profile

Execute the playbook using cgexec

cgexec -g cpuacct,memory,pids:ansible_profile ansible-playbook x.yaml

Analyse the usage

Memory Execution Maximum: 11146.29MB
cpu Execution Maximum: 112.08%
pids Execution Maximum: 35.00
memory:
lab : creating network (b42e9945-0dc7-20b1-09d1-00000000000a): 11097.35MB …..

Enable Task and Role Profiling

The following can be enabled as well, to help debugging playbooks and roles:

 [defaults]
callback_whitelist = profile_tasks, profile_role, timer

Timer: duration of playbook execution (activated by default) profile_tasks/role: displays execution time per tasks/role

This will generate an output like the following:

Directory Structure

A consistent directory structure is important to easily understand all playbooks and roles which are written. Ansible knows many different folder structures, any can be used. However, it is important to stick to one structure.

The following is an example. Not all folders are usually used and working with collections will change such structure a little bit.

.
├── ansible.cfg
├── ansible_modules
├── group_vars
│ ├── webservers
│ └── all
├── hosts
│ ├── webserver01
│ └── webserver02
├── host_vars
├── modules
├── playbooks
│ └── ansible-cmdb.yml
└── roles
├── requirements.yml
├── galaxy
└── dev-sec.ssh-hardening
└── auditd
├── files
│ ├── auditd.conf
│ ├── audit.yml
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
└── tasks
└── main.yml

Automation Controller and LDAP Authentication

Mon, 25 Oct 2021 00:00:00 +0000

The following article shall quickly, without huge background information, deploy an Identity Management Server (based on FreeIPA) and connect this IDM to an existing Automation Controller so authentication can be tested and verified based on LDAP.

Install FreeIPA

Run the following command to deploy and configure the IPA Server:

yum module enable idm:DL1
yum distro-sync
yum -y module install idm:DL1/server
Install the server by calling the command ipa-server-install. This will start an interactive installation modus which requires the basic information about the IPA server. The following uses tower.local as base domain

Do you want to configure integrated DNS (BIND)? [no]:
Server host name [node01.tower.local]:
Please confirm the domain name [tower.local]:
Please provide a realm name [TOWER.LOCAL]:
Directory Manager password: <enter password>
Password (confirm): <enter password>
IPA admin password: <enter password>
Password (confirm): <enter password>
Do you want to configure chrony with NTP server or pool address? [no]:
Continue to configure the system with these values? [no]: yes

Once all information have been provided the installation/configuration process starts. This will take a while…

Be sure that the hostname, here node01.tower.local, is resolvable, at least from the Tower/Controller node and the node you are accessing the FreeIPA UI. You can use your local hosts file or a real domain name for that.

Login to IPA server via Command Line

For user admin use: kinit admin

Create a Binduser (BindDN)

The Binduser (or BindDN) will be used by the Controller to authenticate the Controller against the LDAP server.

Create the actual user

ipa user-add --first=”BindUser” --last=”None” --password binduser

Output:

Password:
Enter Password again to verify:
------------------
Added user "binduser"
------------------
User login: binduser
First name: ”BindUser”
Last name: ”None”
Full name: ”BindUser” ”None”
Display name: ”BindUser” ”None”
Initials: ””
Home directory: /home/binduser
GECOS: ”BindUser” ”None”
Login shell: /bin/sh
Principal name: binduser@TOWER.LOCAL
Principal alias: binduser@TOWER.LOCAL
User password expiration: 20211015133112Z
Email address: binduser@tower.local
UID: 1573400003
GID: 1573400003
Password: True
Member of groups: ipausers
Kerberos keys available: True

Assign the new user to the admin group
```
ipa group-add-member admins --users=binduser
```
Output:

 Group Name: admins
Description: Account administrators group
GID: 1573400000
Member users: admin, binduser
-----------------------------------
Number of members added 1
-----------------------------------

Create a 2nd User to test the authentication later

ipa user-add --first=”User” --last=”Name” --password user1

Enable LDAP Auth in Automation Controller

Login to Automation Controller ad go to "Settings > LDAP Settings > Default"
add a new connection:
1. LDAP Service URI: ldap://node01.tower.local:389
2. LDAP Bind Password: <password of user binduser>`
3. LDAP Group Type: MemberDNGroupType
4. LDAP Bind DN: uid=binduser,cn=users,cn=accounts,dc=tower,dc=local
5. LDAP User Search:
  [ "cn=users,cn=accounts,dc=tower,dc=local", "SCOPE_SUBTREE", "(uid=%(user)s)" ]
6. LDAP Group Search:
  [ "cn=groups,cn=accounts,dc=tower,dc=local", "SCOPE_SUBTREE", "(objectClass=posixgroup)" ]

The configuration should look like the following image:

Figure 1. Automation Controller LDAP Authentication

Verify Login with user1

You can now test the login using user1. If it does not work, check the following files for errors:

Tower Node: /var/log/tower/tower.log

IPA Node: /var/log/dirsrv/slapd-TOWER-LOCAL/access

The login should work, but since the user1 is not assigned to any Team/Organization inside the Automation Controller, no privileges are granted. The user can do nothing.

Automatically assign permissions

2 roles can be automatically assigned to authenticated users:

Super User
Auditor

To test this, 2 groups will be created in the LDAP server and a new user will be assigned to one of the groups.

Create the group for super users: ipa group-add tower_administrators
Create the group for auditors: ipa group-add tower_auditors
Create a new user: ipa user-add --first=”User” --last=”Name” --password user2
Assign the user to one the the groups: ipa group-add-member tower_administrators --users=user2

Modify the Controller LDAP configuration and set LDAP User Flags by Group. This will assing any member of tower_administrators to is_superuser for example.

{
"is_superuser": [
"cn=tower_administrators,cn=groups,cn=accounts,dc=tower,dc=local"
],
"is_system_auditor": [
"cn=tower_auditors,cn=groups,cn=accounts,dc=tower,dc=local"
]
}

Test the authentication and authorization with the user2. This user should now gain super admin permissions.

Allow Users From Specific Groups Only

Not all LDAP users shall be able to authenticate. Only users, which are member of a specific group, shall be able to authenticate.

Create a 3rd user: ipa user-add --first=”User” --last=”Name” --password user3
Modify the LDAP Configuration in Automation Controller and set LDAP Require Groups:
```
"cn=towerusers,cn=groups,cn=accounts,dc=tower,dc=local"
```
Add the group toweruser: ipa group-add towerusers
Assign the user user3 to that group: ipa group-add-member towerusers --users=user3

At this state only user3 will be able to login. In order to allow the other users as well, all must be assigned to the group towerusers

ipa group-add-member towerusers --users=user3
ipa group-add-member towerusers --users=user1

Additional Configuration

It is possible to automatically map users to Controller Organization. I did not fully test this, but the following is an example:

 {
"LDAP Organization": {
"admins": "cn=engineering_admins,ou=groups,dc=example,dc=com",
"remove_admins": false,
"users": [
"cn=engineering,ou=groups,dc=example,dc=com",
"cn=sales,ou=groups,dc=example,dc=com",
"cn=it,ou=groups,dc=example,dc=com"
],
"remove_users": false
},
"LDAP Organization 2": {
"admins": [
"cn=Administrators,cn=Builtin,dc=example,dc=com"
],
"remove_admins": false,
"users": true,
"remove_users": false
}
}

Ansible Tower and downloading collections

Sat, 31 Jul 2021 00:00:00 +0000

Every wondered why Ansible Tower does not start downloading required collections when you synchronize a project? Here are the stumbling blocks we discovered so far:

Wrong name for requirements.yml

When downloading collections Ansible Tower searches for a file requirements.yml in the collections directory.

Be careful with the file extension: requirements.yml has to end with the extension .yml and not .yaml.

Collections download is disabled in Ansible Tower

Within Ansible Tower there is a setting called ENABLE COLLECTION(S) DOWNLOAD under Settings/Jobs. This has to be set to true, which is also the default.

No Ansible Galaxy credential defined for the organization

Last but not least an Ansible Galaxy credential needs to be defined for the organization where the project is defined. With the default installation of Ansible Tower, when the sample playbooks are installed there is a credential called Ansible Galaxy defined. You need to assign this credential to the organization.

If you skip installing the sample playbooks, no Ansible Galaxy credential will be defined for you and you have to create it manually.

How does this actually work?

Ansible Tower uses a Python virtual environment for running Ansible. The default environment is installed in /var/lib/awx/venv/awx. You can also create custom environments, see Using virtualenv with Ansible Tower.

In the default setup the following files define how collections are downloaded:

lib/python3.6/site-packages/awx/main/tasks.py
lib/python3.6/site-packages/awx/playbooks/project_update.yml

task.py

task.py defines various internal tasks Tower has to run on various occasions. For example in line number 1930 (Ansible Tower 3.8.3) the task RunProjectUpdate gets defined. This is the task Tower has to run whenever a project update is required.

In our case the function build_extra_vars_file (line 2083 with Ansible Tower 3.8.3) defines the variable galaxy_creds_are_defined only if the organization has a galaxy credential defined (line 2099 Ansible Tower 3.8.3).

Line 2120 (Ansible Tower 3.8.3) finally defines the Ansible extra variable collections_enabled depending on galaxy_creds_are_defined.

project_update.yml

So task.py defines the extra variable collections_enabled (see above). Finally the playbook project_update.yml consumes this extra variable and only downloads collections if collections_enabled is set to true, see the block string at line 192 (Ansible Tower 3.8.3) in `project_update.yml.

So long and thanks for all the fish!

Ansible - Azure Resource Manager Example

Mon, 27 Apr 2020 00:00:00 +0000

Using Ansible Resource Manager with an ARM template and a simple Ansible playbook to deploy a Virtual Machine with Disk, virtual network, public IP and so on.

Introduction

Source: [1]

In order to deploy a Virtual Machine and all depended resources using the Azure Resource Manager (ARM) template with Ansible, you will need three things:

The ARM Template
The parameters you want to use
The Ansible playbook

All can be found below. Store them and simply call:

ansible-playbook Azure/create_azure_deployment.yml

it will take several minutes, until everything has been deployment in Azure.

Instead of using a json file locally you can upload the template file (as well as a parameters file) to a version control system and use it from there.

Additional Resources

Ansible Playbook: Create-Azure-Deplyoment.yml

---
- name: Get facts of a VM
hosts: localhost
connection: local
become: false
gather_facts: false
tasks:
#- name: Destroy Azure Deploy
# azure_rm_deployment:
# resource_group: tju-ResourceGroup
# name: tju-testDeployment
# state: absent
- name: Create Azure Resource Group deployment
azure_rm_deployment:
state: present
resource_group_name: tju-ResourceGroup
name: tju-testDeployment
#template_link: '<YOUR RAW Github template file>'
#parameters_link: '<YOUR RAW Github parameters file>'
template: "{{ lookup('file', 'ResourceManagerTemplate.json') }}"
parameters:
projectName:
value: tjuProject
location:
value: "East US"
adminUsername:
value: tjungbauer
adminPublicKey:
value: "{{ lookup('file', '/Users/tjungbauer/.ssh/id_rsa.pub') }}"
operatingSystem:
value: CentOS
operatingSystemPublisher:
value: OpenLogic
operatingSystemSKU:
value: '7.1'
vmSize:
value: Standard_D2s_v3
register: azure
- name: Add new instance to host group
add_host:
hostname: "{{ item['ips'][0].public_ip }}"
groupname: azure_vms
loop: "{{ azure.deployment.instances }}"

ResourceManagerTemplate.json

{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"projectName": {
"type": "string",
"metadata": {
"description": "Specifies a name for generating resource names."
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Specifies the location for all resources."
}
},
"adminUsername": {
"type": "string",
"metadata": {
"description": "Specifies a username for the Virtual Machine."
}
},
"adminPublicKey": {
"type": "string",
"metadata": {
"description": "Specifies the SSH rsa public key file as a string. Use \"ssh-keygen -t rsa -b 2048\" to generate your SSH key pairs."
}
},
"operatingSystem": {
"type": "string",
"metadata": {
"description": "Specifies the Operating System. i.e. CentOS"
}
},
"operatingSystemPublisher": {
"type": "string",
"metadata": {
"description": "Specifies the publisher. i.e. OpenLogic"
}
},
"operatingSystemSKU": {
"type": "string",
"metadata": {
"description": "Specifies the version of the OS. i.e. 7.1"
}
},
"vmSize": {
"type": "string",
"metadata": {
"description": "Specifies the the VM size. i.e. Standard_D2s_v3"
}
}
},
"variables": {
"vNetName": "[concat(parameters('projectName'), '-vnet')]",
"vNetAddressPrefixes": "10.0.0.0/16",
"vNetSubnetName": "default",
"vNetSubnetAddressPrefix": "10.0.0.0/24",
"vmName": "[concat(parameters('projectName'), '-vm')]",
"publicIPAddressName": "[concat(parameters('projectName'), '-ip')]",
"networkInterfaceName": "[concat(parameters('projectName'), '-nic')]",
"networkSecurityGroupName": "[concat(parameters('projectName'), '-nsg')]",
"networkSecurityGroupName2": "[concat(variables('vNetSubnetName'), '-nsg')]"
},
"resources": [
{
"type": "Microsoft.Network/networkSecurityGroups",
"apiVersion": "2018-11-01",
"name": "[variables('networkSecurityGroupName')]",
"location": "[parameters('location')]",
"properties": {
"securityRules": [
{
"name": "ssh_rule",
"properties": {
"description": "Locks inbound down to ssh default port 22.",
"protocol": "Tcp",
"sourcePortRange": "*",
"destinationPortRange": "22",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "*",
"access": "Allow",
"priority": 123,
"direction": "Inbound"
}
}
]
}
},
{
"type": "Microsoft.Network/publicIPAddresses",
"apiVersion": "2018-11-01",
"name": "[variables('publicIPAddressName')]",
"location": "[parameters('location')]",
"properties": {
"publicIPAllocationMethod": "Dynamic"
},
"sku": {
"name": "Basic"
}
},
{
"comments": "Simple Network Security Group for subnet [variables('vNetSubnetName')]",
"type": "Microsoft.Network/networkSecurityGroups",
"apiVersion": "2019-08-01",
"name": "[variables('networkSecurityGroupName2')]",
"location": "[parameters('location')]",
"properties": {
"securityRules": [
{
"name": "default-allow-22",
"properties": {
"priority": 1000,
"access": "Allow",
"direction": "Inbound",
"destinationPortRange": "22",
"protocol": "Tcp",
"sourceAddressPrefix": "*",
"sourcePortRange": "*",
"destinationAddressPrefix": "*"
}
}
]
}
},
{
"type": "Microsoft.Network/virtualNetworks",
"apiVersion": "2018-11-01",
"name": "[variables('vNetName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName2'))]"
],
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('vNetAddressPrefixes')]"
]
},
"subnets": [
{
"name": "[variables('vNetSubnetName')]",
"properties": {
"addressPrefix": "[variables('vNetSubnetAddressPrefix')]",
"networkSecurityGroup": {
"id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName2'))]"
}
}
}
]
}
},
{
"type": "Microsoft.Network/networkInterfaces",
"apiVersion": "2018-11-01",
"name": "[variables('networkInterfaceName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/publicIPAddresses', variables('publicIPAddressName'))]",
"[resourceId('Microsoft.Network/virtualNetworks', variables('vNetName'))]",
"[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]"
],
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('publicIPAddressName'))]"
},
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('vNetName'), variables('vNetSubnetName'))]"
}
}
}
]
}
},
{
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2018-10-01",
"name": "[variables('vmName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/networkInterfaces', variables('networkInterfaceName'))]"
],
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('vmSize')]"
},
"osProfile": {
"computerName": "[variables('vmName')]",
"adminUsername": "[parameters('adminUsername')]",
"linuxConfiguration": {
"disablePasswordAuthentication": true,
"ssh": {
"publicKeys": [
{
"path": "[concat('/home/', parameters('adminUsername'), '/.ssh/authorized_keys')]",
"keyData": "[parameters('adminPublicKey')]"
}
]
}
}
},
"storageProfile": {
"imageReference": {
"publisher": "[parameters('operatingSystemPublisher')]",
"offer": "[parameters('operatingSystem')]",
"sku": "[parameters('operatingSystemSKU')]",
"version": "latest"
},
"osDisk": {
"createOption": "fromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('networkInterfaceName'))]"
}
]
}
}
}
],
"outputs": {
"adminUsername": {
"type": "string",
"value": "[parameters('adminUsername')]"
}
}
}

Sources

[1]: azure_rm_deployment – Create or destroy Azure Resource Manager template deployments

DO410 Ansible and Ansible Tower training notes

Mon, 06 Apr 2020 00:00:00 +0000

Notes taken during Red Hat course D410 Ansible and Ansible Tower.

Ansible installation

make sure that libselinux-python is installed
Ansible 2.7 requires python 2.6 or 3.5

yum list installed python

windows modules implemented in powershell
ansible requires at least .net 4.0

Configuration files

Ansible searches for ansible.cfg in the following order:

$ANSIBLE_CFG
ansible.cfg in the current directory
$HOME/ansible.cfg
/etc/ansible/ansible.cfg

whichever it finds first will be used.

use

ansible --version

to see which config file is currently used. you can view/dump/see what changed with

ansible-config [list|dump|view]

Default modules

List all available modules via

ansible-doc -l

For getting help on a specific module use

ansible-doc ping

Ad-hoc commmands

To display ansible output on a single line per host for easier readablility use the -o option

ansible all -m command -a /bin/hostname -o

Use the raw module for directly executing commands on remote systems that do not have python installed.

ansible -m raw

Custom Facts

Ansible uses custom facts from /etc/ansible/facts.d/. Facts can be stored in .ini style or you can place executable scripts in this directory. The script needs to output JSON. Custom facts are available via ansible_facts.ansible_local.

Magic variables available

hostvars: variables defined for this host
group_names: list of groups this host is a member of
groups: list of all groups and hosts in the inventory
inventory_hostname: host name of the current host as configured in the inventory

Matching hosts in the inventory

Some examples on how to match hosts defined in the inventory

'*.lab.com': match all hosts starting with lab.com
'lab,datacenter': match all hosts either in lab or datacenter
'datacenter*': match all host and host groups starting with datacenter
'lab,&datacenter': match hosts in the lab and datacenter group
'datacenter,!test.lab.com': match all hosts in datacenter, except test.lab.com

Dynamic inventory

Example scripts for dynamic inventories can be found at https://github.com/ansible/ansible/tree/devel/contrib/inventory.

You can use ansible-inventory to take a look a the current inventory as json. This also works for static inventories.

Inventories can be combined. Just create a directory containing a static inventory and script to create a dynamic inventory, ansible will happily execute the scripts and merge everything together.

Debugging

The following might be useful when debugging ansible roles and playbooks

ansible-playbook play.yml --syntax-check
ansible-playbook play.yml --step
ansible-playbook play.yml --start-at-task="start httpd service"
ansible-playbook --check play.yml
ansible-playbook --check --diff play.yml

Ansible Tower

Notes on deploying and working with ansible tower.

Installation

System requirements:

at least 4GB of RAM
actual requirement depends on forks variable
recommendation is 100MB memory for each for + 2GB of memory for tower services
20GB of disk storage, at least 10GB in /var

Steps for installing:

download setup tar.gz from http://releases.ansible.com/ansible-tower/setup/
set passwords in inventory
run ./setup.sh

Authentication

Authentication settings can be changed under Settings / Authentication. E.g for configuring Azure AD authentication we are going to need

an Azure AD oauth2 key and
a Azure AD oauth2 secret

RBAC

separate roles for organizations and inventories
you need to assign roles to organizations and inventories

The Tower Flow

These are the steps to run playbooks against managed nodes in Tower:

Create an organization if required
Create users
Create teams and assign users
Create credentials for accessing managed nodes
Assign credential to organization
Create credentials for accessing SCM repositories (e.g. git)
Assign credentials to users or teams
Create a project
Assign Teams to project
Create a job template for executing playbooks

Ansible Roles support

If the project includes a requirements.txt file in the roles/ folder, tower will automatically run

ansible-galaxy install -r roles/requirements.yml -p ./roles/ --force

at the end of an update. So this could be used to include external dependencies (like SAP ansible roles).

Job Templates

Ansible playbooks are stored in GIT repositories. A job template defines

the inventory used for this job template
the project for executing this job
this connects the GIT repository used in this project with the template
the playbook to execute
the credentials for executing jobs
permissions for users / teams (e.g. admin, execute)

Tower creates jobs from those templates, which are ansible runs executed against managed nodes.

Fact Caching

It might be a good idea to use the tower facts cache. To speed up playbook runs set gather_facts: no in the play. Then enable the facts cache in tower.

In tower settings set a timeout for the cache
In job templates enable Use facts cache
Create a playbook that runs on a regular basis to gather facts, e.g.

- name: Refresh fact cache
hosts: all
gather_facts: yes

Inventory options

These are the options for creating inventories in Ansible Tower

static inventory defined in tower
importing static inventories via awx-manage
static inventory defined in git repository
dynamic inventory via a custom script
dynamic inventory provides by tower (e.g. satellite)

A special feature in Tower are so called smart inventories. A smart inventory combines all static and dynamic inventories and allows filtering based on facts. Filtering requires a valid fact cache.

Troubleshooting

Tower uses the following components:

postgresql
nginx
memcached
rabbitmq
supervisord

Useful tools

ansible-tower-service (e.g. status / restart)
supervisorctl (e.g. status)
awx-manage

Tower stores log files in

/var/log/tower/ (e.g. tower.log).
/var/log/supervisor/
/var/log/nginx/

Other important directories

/var/lib/awx/public/static static files served by django
/var/lib/awx/projects stores all project related files e.g. git checkouts)
/var/lib/awx/jobs_status job status output

by default playbook runs are confined to /tmp this might lead to problems with tasks running on the local system.

In case of a lost admin password you can use awx-manage to reset the password or create a new superuser:

awx-manage changepassword admin
awx-manage createsuperuser

Replacing the default TLS certificates

Ansible tower uses nginx to service it’s web interface over TLS. Nginx uses the configuration file /etc/nginx/nginx.conf.

To deploy custom TLS certificates used by tower replace the certificate and private key in /etc/tower. You have to replace

/etc/tower/tower.crt and
/etc/tower/tower.key

It might be a good idea to create a backup copy before overwriting those files.

Backup and restore

Of course backup and restore are done via ansible. The ansible tower setup script setup.sh provides a wrapper around these playbooks. Execute

setup.sh -b

to perform a backup. This creates a backup .tar.gz file in the current directory.

To restore a backup use

setup.sh -r

this restores the latest backup per default.

Things to remember

Workflow job templates
add autocmd FileType yaml setlocal ai ts=2 sw=2 et to .vimrc
use sudo yum install python-cryptography if there are many vault files to speed up ansible