Ansible playbooks are better left empty


The code

The associated repository and Ansible playbook can be found here:

https://github.com/cidrblock/example_empty_playbook

The problem

As the use of Ansible to manage the network grows within our organization, the number of roles and complexity of the data is ever increasing. We need decrease the size of our Ansible playbooks in an effort to flatten the complexity curve to a manageable level.

  • Roles and templates become dependent on a specific data structure within the inventory.

  • The inventory is growing more complex as purpose built groups are added to target subsets of the fleet that share a common configuration.

  • The number of internal modules, lookup filters, and filter plugins is increasing as well.

  • The reusability of roles is increasing, time is wasted when two network engineers author the same role for different changes for use in different playbooks.

Acting like software developers

Combining development versioning and release practices with native Ansible functionality can provide some relief and lock the data model to the roles to the plugins to a specific version for compatibility. A playbook should specifically reference the data, role, and plugin versions for a successful run.

Build empty playbooks

Take the following playbook as an example: https://github.com/cidrblock/identify_port_profiles_across_devices. It is a complete standalone playbook. The inventory, plugins, and roles are all contained within the repository. There is a snake in the grass though, a classic mistake I suspect we have all made. If the configuration of the network is defined across playbooks in separate inventory files, we have simply moved the distributed configuration from the network into git.

Looking across the enterprise network, configuration drift can be found as "standards" and software versions have changed. Taking advantage of "a better way to do this" has come at a cost because the fleet has not always been updated to reflect the version and configuration of the latest install.

Roles will expect a particular data model, the data model will have "improved" and what was expected to be a simply change will involve hours of retrofitting roles and jinja templates. Technical debt will slowly chip away at the effeciencies automation bought us and reverting to CTRL-A, CTRL-C, ssh, CTRL-V will be the way to get it done fast. The current inconsistencies across the fleet will be moved from the devices into Ansible, git, the data, and code if playbooks aren't kept empty.

Yes, keep the Ansible playbooks empty.

The hierarchy of a playbook

The major components of an playbook include:

  • roles: "Roles in Ansible build on the idea of include files and combine them to form clean, reusable abstractions – they allow you to focus more on the big picture and only dive down into the details when needed."

  • plugins: "Plugins are pieces of code that augment Ansible’s core functionality. Ansible ships with a number of handy plugins, and you can easily write your own."

  • inventory: "Ansible works against multiple systems in your infrastructure at the same time. It does this by selecting portions of systems listed in Ansible’s inventory file."

Let's explore one way to disassemble the playbook into smaller pieces, each having it's own version and git repository. The playbook will assemble the necessary roles, plugins, inventory at runtime.

Roles

Roles can be either added to Ansible galaxy or hosted on premise in an internal installation of git or github enterprise.

The dependencies, the roles added to the playbook runtime can be referenced by adding each to a requirements.yml file:

- name: company_plugins
  src: https://github.com/cidrblock/role_company_plugins
  version: '1.01'

- name: ansible_change_report
  src: https://github.com/cidrblock/role_ansible_change_report
  version: '1.01'

- name: interface
  src: https://github.com/cidrblock/role_interface
  version: '1.01'

Note the specific tag for each. This allows the compatible versions to be specified for the playbook.

Running ansible-galaxy install -r requirements.yml -p roles would install the roles to the roles directory in the playbook.

Inventory

We currently use a combination of static and dynamic inventory files. A subset of the inventory data needed can be harvested from our on premise monitoring tool REST API. The inventory can be kept in it's own git repository as well. The inventory can be versioned as well, but each version would need to be maintained with current data to avoid a massive roll-back. Tackling this specific problem will need additional thought and process.

Collecting the inventory:

git clone -b 1.01 https://github.com/cidrblock/company_inventory inventory

Plugins

Rather than keeping plugins in the root of the Ansible playbook directory they can be contained within a role. As long as the plugin role is run first, the plugins will be available to subsequent roles: reference

The lookup and filter plugins from the examples above have been moved into a role. https://github.com/cidrblock/role_company_plugins

And can be run as the first role:

- hosts: demo_hosts
  gather_facts: no
  connection: local
  roles:
  - company_plugins

Putting it all together

Here's the new version of the playbook. https://github.com/cidrblock/example_empty_playbook

  • A .gitignore hase been added to prevent the inventory or roles from being checked in with the playbook's source.

  • A requirements.yml file has been added and references the dependent roles, including the plugin role.

  • For convenience, an assemble.sh script which runs the clone and ansible-galaxy command, prompts to kick off the playbook run in check mode.

  • Roles, plugins, and inventory have been removed.

#! /bin/bash
git clone -b 1.01 https://github.com/cidrblock/company_inventory inventory
ansible-galaxy install -r requirements.yml -p roles
ansible-playbook -i inventory site.yml --list-host
ansible-playbook -i inventory site.yml --list-tasks
read -p "Run playbook in check mode (y/n)?" choice
case "$choice" in
  y|Y ) ansible-playbook -i inventory site.yml --check;;
  n|N ) echo "exiting";;
  * ) echo "invalid";;
esac

Not completely empty but close, we're down to three files.

Sample run

➜  /working git:(master) tree
.
├── assemble.sh
├── requirements.yml
└── site.yml

0 directories, 3 files
➜  /working git:(master) ./assemble.sh
Cloning into 'inventory'...
remote: Counting objects: 15, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 15 (delta 3), reused 15 (delta 3), pack-reused 0
Unpacking objects: 100% (15/15), done.
Note: checking out '12aa4b507f2ef236077073347073d92ab116b0b1'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

- extracting company_plugins to roles/company_plugins
- company_plugins was installed successfully
- extracting ansible_change_report to roles/ansible_change_report
- ansible_change_report was installed successfully
- extracting interface to roles/interface
- interface was installed successfully

playbook: site.yml

  play #1 (demo_hosts): demo_hosts  TAGS: []
    pattern: [u'demo_hosts']
    hosts (2):
      xe_int_sample_02
      xe_int_sample_01

playbook: site.yml

  play #1 (demo_hosts): demo_hosts  TAGS: []
    tasks:
      set_fact  TAGS: []
      interface : Update the interface with their parents   TAGS: []
      interface : Include OS files for interfaces   TAGS: []
      ansible_change_report : Show config changes for device    TAGS: []
Run playbook in check mode (y/n)?y

PLAY [demo_hosts] **************************************************************
<...>
PLAY RECAP *********************************************************************
xe_int_sample_01           : ok=504  changed=0    unreachable=0    failed=0
xe_int_sample_02           : ok=504  changed=0    unreachable=0    failed=0

➜  /working git:(master)

Wrap up

We are network engineers striving and learning to become software developers in an effort to eliminate the toil. As our understanding of what it means to automate every aspect of the network matures so will our code revision and control processes. Thankfully many others have already traveled this path and are far ahead of us. Needing to start someplace to avoid the pitfalls in our history, this seems reasonable.

I'm open to suggestions.