Skip to main content

Testing Ansible roles with Molecule

I recently heard about Molecule, a tool for testing roles in Ansible. Essentially the whole of the conversation was:
"Have you ever used Molecule?"
"No - what's that?"
"It's for testing Ansible roles. I haven't used it either, but someone told me it was cool."
I went about reading the docs, and felt like they were not great for someone who had no idea what was going on (the API documentation is great, though). After doing some playing around with it, I think Molecule looks pretty useful, so here is what I wish the docs had told me up front.

What is it?

Molecule provides a lot of the boilerplate required to test Ansible roles. It provides tools to automagically stand up test hosts using Docker, Vagrant, LXD, Azure and many other virtualization and containerization providers, lint and run the role to be tested, and run Goss or Testinfra tests against the test hosts.

Getting started

You can install Molecule using pip:

pip install molecule

By default, Molecule uses the Docker provider. This is probably the simplest and fastest to get started with - to use it you need Docker Engine installed, and the docker-py Python package:

pip install docker-py

Now you're ready to create a new role using molecule (we'll call it nginx):

molecule init role -r nginx

This will create a new role, much like if you'd used ansible-galaxy init, along with all the Molecule goodies, all of which are stored under a directory named molecule:
ben@munin:~/dev/github/bengerman13/molecule-testing/roles/nginx$ tree molecule/
molecule/
└── default
    ├── create.yml
    ├── destroy.yml
    ├── Dockerfile.j2
    ├── INSTALL.rst
    ├── molecule.yml
    ├── playbook.yml
    ├── prepare.yml
    └── tests
        ├── test_default.py
        └── test_default.pyc

2 directories, 9 files

If this all worked correctly, you should be able to test your (empty) role by running molecule test from inside your role directory. Molecule will then:

  1. Lint all of the .yml and .py files under the nginx directory, using Yamllint for yaml and flake8 for python
  2. Destroy any existing test hosts for this role. It does this by calling the playbook molecule/default/destroy.yml
  3. Downloads dependent roles from Ansible Galaxy
  4. Create a suitable test host. This is done by calling the playbook molecule/default/create.yml, which will use the Dockerfile template (Dockerfile.j2) to build a Docker container (if it doesn't exist already), then use the Ansible Docker module to start the container on your machine
  5. Run prepare.yml against your test host
  6. Run playbook.yml against your test host
  7. Run playbook.yml against your test host a second time
  8. Run the tests found under molecule/default/tests
  9. Destroy the test hosts

Digging Deeper

In order to go any further, I think it's necessary to explain the structure Molecule follows. Each role has what are called scenarios, and every role has a 'default' scenario. That's where the 'default' directory comes in. Scenarios define 'actions', which are what commands actually run. When we ran molecule test above, we were running the test command against the default scenario. The test command runs these actions, in order:
  • lint
  • destroy
  • dependency
  • syntax
  • create
  • prepare
  • converge - run your role a first time
  • idempotence - run your role a second time, and fail if any tasks report change
  • side_effect - optional: this allows you to then manipulate your host(s) somehow to test its behavior, for instance testing failover in a multihost environment
  • verify - runs the tests in tests directory
  • destroy
Scenarios also define the dependency manager (i.e. Ansible Galaxy or Gilt), driver (i.e. Docker, ec2), yaml linter, provisioner (ansible is the only provisioner), verifier (Testinfra or Goss), and platforms. Platforms and driver are probably the most interesting things for most users staying on the beaten path. Platforms are the actual machines to test your role against, and the way you define them is somewhat driver-dependent. By default, a single platform is defined for us, based on the centos:7 Docker image:

platforms:
  - name: instance
    image: centos:7

When the create action is executed, it checks to see if it needs to create or update the Docker image to be used, and builds the image as necessary. It then creates a Docker container from the image:

(molecule) ben@munin:~/dev/github/bengerman13/molecule-testing/roles/nginx$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
(molecule) ben@munin:~/dev/github/bengerman13/molecule-testing/roles/nginx$ molecule create
--> Test matrix
    
└── default
    ├── create
    └── prepare
    
--> Scenario: 'default'
--> Action: 'create'
    
    PLAY [Create] ******************************************************************
    
    TASK [Create Dockerfiles from image names] *************************************
    ok: [localhost] => (item=None)
    
    TASK [Discover local Docker images] ********************************************
    ok: [localhost] => (item=None)
    
    TASK [Build an Ansible compatible image] ***************************************
    ok: [localhost] => (item=None)
    
    TASK [Create molecule instance(s)] *********************************************
    ok: [localhost] => (item=None)
    
    TASK [Wait for instance(s) creation to complete] *******************************
    ok: [localhost] => (item=None)
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=5    changed=3    unreachable=0    failed=0
    
    
--> Scenario: 'default'
--> Action: 'prepare'
    
    PLAY [Prepare] *****************************************************************
    
    PLAY RECAP *********************************************************************
    
    
(molecule) ben@munin:~/dev/github/bengerman13/molecule-testing/roles/nginx$ docker ps
CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS              PORTS               NAMES
c921512db674        molecule_local/ubuntu:18.04   "bash -c 'while tr..."   9 seconds ago       Up 7 seconds                            instance

Example

Lets make our nginx role, and do some light testing on it.
First, we'll create the role using molecule:

molecule init role -r nginx

For our testing, let's change the image to Ubuntu 16.04 by editing molecule/default/molecule.yml:

...
platforms:
  - name: instance
    image: ubuntu:16.04
...

Our role should install nginx, start and enable the service.Here's nginx/tasks/main.yml:

---
# tasks file for nginx
- name: Install nginx
  apt:
    name: nginx
    state: installed
  notify: Start nginx

and handlers/main.yml:
---
# handlers file for nginx

- name: Start nginx
  command: /usr/sbin/nginx

We can run molecule test to show that we haven't broken anything, but we are not yet really testing that our playbook does what we want. That's where Testinfra (and/or Goss) come into play.

Testinfra

In our simple role, we wanted to do two things - install nginx and start nginx. Let's test that we accomplished both. Create a new file nginx/molecule/default/tests/test_nginx.py:

1
2
3
4
5
6
7
8
9
def test_nginx_installed(host):
"""Make sure nginx is installed."""
    assert host.package('nginx').is_installed


def test_nginx_running(host):
"""Make sure nginx is running."""
    procs = host.process.filter(user='root', comm='nginx')
    assert len(procs)

Running the molecule test again shows that we have indeed installed and started nginx.
What if we want to make sure nginx is really listening? We could add this to molecule/default/test_nginx.py:

11
12
def test_nginx_is_listening(host):
    assert host.socket("tcp://80").is_listening

Run our tests again and... the new test fails.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
    ------------------------------ Captured log call -------------------------------
    ansible.py                  64 INFO     RUN Ansible(u'shell', u'netstat -n -l -t', {}): {'_ansible_no_log': False,
     '_ansible_parsed': True,
     u'changed': True,
     u'cmd': u'netstat -n -l -t',
     u'delta': u'0:00:00.096925',
     u'end': u'2017-12-03 08:02:01.696335',
     u'invocation': {u'module_args': {u'_raw_params': u'netstat -n -l -t',
                                      u'_uses_shell': True,
                                      u'chdir': None,
                                      u'creates': None,
                                      u'executable': None,
                                      u'removes': None,
                                      u'stdin': None,
                                      u'warn': True}},
     u'msg': u'non-zero return code',
     u'rc': 127,
     u'start': u'2017-12-03 08:02:01.599410',
     u'stderr': u'/bin/sh: 1: netstat: not found',
     'stderr_lines': [u'/bin/sh: 1: netstat: not found'],
     u'stdout': u'',
     'stdout_lines': []}
    base.py                    241 INFO     RUN CommandResult(command=u'netstat -n -l -t', exit_status=127, stdout=u'', stderr=u'/bin/sh: 1: netstat: not found')
    ====================== 1 failed, 3 passed in 7.82 seconds ======================

Checking sockets requires netstat, which is not included in this Docker image. We don't want to add netstat as a dependency of our role, since it's not really required for nginx - this is where the prepare step comes in. Edit molecule/default/prepare.yml:

---
- name: Prepare
  hosts: all
  gather_facts: false
  tasks:
    - name: Install netstat
      apt:
        name: net-tools
        state: installed

Running molecule test again, all of the tests pass.
One thing to be aware of here is that prepare happens before the role is run. This means it's possible to accidentally hide real role dependencies from yourself. Another option would be to install net-tools in the side-effect action, but that does not seem idiomatic to me (also, side-effects are noted to be an unstable interface in the docs). Yet another option would be to add a task to molecule/default/playbook.yml, which might be the best option.

But I already wrote my role!

It's pretty easy to add molecule testing to an existing role. From inside your role's directory, run molecule init scenario -r role_name then start writing your tests.

Final Thoughts

Molecule greatly simplifies the testing setup for ansible roles, leaving little excuse to not test roles going forward.
As I mentioned before, the Docker driver makes it quick and easy to get started, but I quickly ran into some limitations, specifically Docker doesn't work well with systemd. To test running services under a service manager, it's helpful to use one of the full VM drivers or lxd.

Comments

Post a Comment

Popular posts from this blog

Who to blame for all your problems

Who to blame for all your problemsConducting Blameless PostmortemsThis post is based off of my talk at PyCascades 2019
To start off with, what is a postmortem?
There are two common uses of the term:A document detailing what happened during an incidentA meeting to review an incident, usually resulting in the creation of the postmortem document
This post is focused on the meeting, but I'll also have some recommendations for the document.Why Run Postmortems?Why do we conduct postmortems, anyway? Production broke, we fixed it, call it a day, right?
Holding postmortems helps us understand better how our systems work -- and how they don't.If your system is complex (and it probably is), the people who work on it have an incomplete and inaccurate view of how it works. Incidents highlight where these gaps and inaccuracies lie. Reviewing incidents after the fact will improve your understanding of your systems. By doing this as a group and sharing what you found, you can improve your who…

JavaScript: array.every()

I decided to relearn JavaScript after not having worked with it since school so I'm going through the JavaScript track at exercism.io. Yesterday, I ran into behavior I didn't expect with Array.prototype.every(), and after getting help from some actual JS developers, they were surprised too.

The Exercism project I was working on was to create a class to check if a sentence is a pangram. (A pangram is a sentence that contains every letter of the alphabet.)

My approach was simple: create a sparse array with 26 elements (one for each letter) walk the sentence, and for each letter, look up its index in the alphabet, then set that index in my array to true. After I was done walking the sentence, I then used Array.prototype.every() to check the results. Array.prototype.every() seemed perfect for this: it takes a callback function, and applies it to every element of an array. If it gets back a falsy return value from one an element, it stops processing and returns false. If it makes i…