Testing Ansible roles with Molecule

I recently heard about Molecule, a tool for testing roles in Ansible. Essentially the whole of the conversation was:
"Have you ever used Molecule?"
"No - what's that?"
"It's for testing Ansible roles. I haven't used it either, but someone told me it was cool."
I went about reading the docs, and felt like they were not great for someone who had no idea what was going on (the API documentation is great, though). After doing some playing around with it, I think Molecule looks pretty useful, so here is what I wish the docs had told me up front.

What is it?

Molecule provides a lot of the boilerplate required to test Ansible roles. It provides tools to automagically stand up test hosts using Docker, Vagrant, LXD, Azure and many other virtualization and containerization providers, lint and run the role to be tested, and run Goss or Testinfra tests against the test hosts.

Getting started

You can install Molecule using pip:

pip install molecule

By default, Molecule uses the Docker provider. This is probably the simplest and fastest to get started with - to use it you need Docker Engine installed, and the docker-py Python package:

pip install docker-py

Now you're ready to create a new role using molecule (we'll call it nginx):

molecule init role -r nginx

This will create a new role, much like if you'd used ansible-galaxy init, along with all the Molecule goodies, all of which are stored under a directory named molecule:

ben@munin:~/dev/github/bengerman13/molecule-testing/roles/nginx$ tree molecule/
molecule/
└── default
    ├── create.yml
    ├── destroy.yml
    ├── Dockerfile.j2
    ├── INSTALL.rst
    ├── molecule.yml
    ├── playbook.yml
    ├── prepare.yml
    └── tests
        ├── test_default.py
        └── test_default.pyc

2 directories, 9 files

If this all worked correctly, you should be able to test your (empty) role by running molecule test from inside your role directory. Molecule will then:

Lint all of the .yml and .py files under the nginx directory, using Yamllint for yaml and flake8 for python
Destroy any existing test hosts for this role. It does this by calling the playbook molecule/default/destroy.yml
Downloads dependent roles from Ansible Galaxy
Create a suitable test host. This is done by calling the playbook molecule/default/create.yml, which will use the Dockerfile template (Dockerfile.j2) to build a Docker container (if it doesn't exist already), then use the Ansible Docker module to start the container on your machine
Run prepare.yml against your test host
Run playbook.yml against your test host
Run playbook.yml against your test host a second time
Run the tests found under molecule/default/tests
Destroy the test hosts

Digging Deeper

In order to go any further, I think it's necessary to explain the structure Molecule follows. Each role has what are called scenarios, and every role has a 'default' scenario. That's where the 'default' directory comes in. Scenarios define 'actions', which are what commands actually run. When we ran molecule test above, we were running the test command against the default scenario. The test command runs these actions, in order:

lint
destroy
dependency
syntax
create
prepare
converge - run your role a first time
idempotence - run your role a second time, and fail if any tasks report change
side_effect - optional: this allows you to then manipulate your host(s) somehow to test its behavior, for instance testing failover in a multihost environment
verify - runs the tests in tests directory
destroy

Scenarios also define the dependency manager (i.e. Ansible Galaxy or Gilt), driver (i.e. Docker, ec2), yaml linter, provisioner (ansible is the only provisioner), verifier (Testinfra or Goss), and platforms. Platforms and driver are probably the most interesting things for most users staying on the beaten path. Platforms are the actual machines to test your role against, and the way you define them is somewhat driver-dependent. By default, a single platform is defined for us, based on the centos:7 Docker image:

platforms:
  - name: instance
    image: centos:7

When the create action is executed, it checks to see if it needs to create or update the Docker image to be used, and builds the image as necessary. It then creates a Docker container from the image:

(molecule) ben@munin:~/dev/github/bengerman13/molecule-testing/roles/nginx$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
(molecule) ben@munin:~/dev/github/bengerman13/molecule-testing/roles/nginx$ molecule create
--> Test matrix
    
└── default
    ├── create
    └── prepare
    
--> Scenario: 'default'
--> Action: 'create'
    
    PLAY [Create] ******************************************************************
    
    TASK [Create Dockerfiles from image names] *************************************
    ok: [localhost] => (item=None)
    
    TASK [Discover local Docker images] ********************************************
    ok: [localhost] => (item=None)
    
    TASK [Build an Ansible compatible image] ***************************************
    ok: [localhost] => (item=None)
    
    TASK [Create molecule instance(s)] *********************************************
    ok: [localhost] => (item=None)
    
    TASK [Wait for instance(s) creation to complete] *******************************
    ok: [localhost] => (item=None)
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=5    changed=3    unreachable=0    failed=0
    
    
--> Scenario: 'default'
--> Action: 'prepare'
    
    PLAY [Prepare] *****************************************************************
    
    PLAY RECAP *********************************************************************
    
    
(molecule) ben@munin:~/dev/github/bengerman13/molecule-testing/roles/nginx$ docker ps
CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS              PORTS               NAMES
c921512db674        molecule_local/ubuntu:18.04   "bash -c 'while tr..."   9 seconds ago       Up 7 seconds                            instance

Example

Lets make our nginx role, and do some light testing on it.

First, we'll create the role using molecule:

molecule init role -r nginx

For our testing, let's change the image to Ubuntu 16.04 by editing molecule/default/molecule.yml:

...
platforms:
  - name: instance
    image: ubuntu:16.04

...

Our role should install nginx, start and enable the service.Here's nginx/tasks/main.yml:

---
# tasks file for nginx
- name: Install nginx
  apt:
    name: nginx
    state: installed
  notify: Start nginx

and handlers/main.yml:

---
# handlers file for nginx

- name: Start nginx
  command: /usr/sbin/nginx

We can run molecule test to show that we haven't broken anything, but we are not yet really testing that our playbook does what we want. That's where Testinfra (and/or Goss) come into play.

Testinfra

In our simple role, we wanted to do two things - install nginx and start nginx. Let's test that we accomplished both. Create a new file nginx/molecule/default/tests/test_nginx.py:

def test_nginx_installed(host):
"""Make sure nginx is installed."""
    assert host.package('nginx').is_installed


def test_nginx_running(host):
"""Make sure nginx is running."""
    procs = host.process.filter(user='root', comm='nginx')
    assert len(procs)

Running the molecule test again shows that we have indeed installed and started nginx.
What if we want to make sure nginx is really listening? We could add this to molecule/default/test_nginx.py:

11
12

def test_nginx_is_listening(host):
    assert host.socket("tcp://80").is_listening

Run our tests again and... the new test fails.

    ------------------------------ Captured log call -------------------------------
    ansible.py                  64 INFO     RUN Ansible(u'shell', u'netstat -n -l -t', {}): {'_ansible_no_log': False,
     '_ansible_parsed': True,
     u'changed': True,
     u'cmd': u'netstat -n -l -t',
     u'delta': u'0:00:00.096925',
     u'end': u'2017-12-03 08:02:01.696335',
     u'invocation': {u'module_args': {u'_raw_params': u'netstat -n -l -t',
                                      u'_uses_shell': True,
                                      u'chdir': None,
                                      u'creates': None,
                                      u'executable': None,
                                      u'removes': None,
                                      u'stdin': None,
                                      u'warn': True}},
     u'msg': u'non-zero return code',
     u'rc': 127,
     u'start': u'2017-12-03 08:02:01.599410',
     u'stderr': u'/bin/sh: 1: netstat: not found',
     'stderr_lines': [u'/bin/sh: 1: netstat: not found'],
     u'stdout': u'',
     'stdout_lines': []}
    base.py                    241 INFO     RUN CommandResult(command=u'netstat -n -l -t', exit_status=127, stdout=u'', stderr=u'/bin/sh: 1: netstat: not found')
    ====================== 1 failed, 3 passed in 7.82 seconds ======================

Checking sockets requires netstat, which is not included in this Docker image. We don't want to add netstat as a dependency of our role, since it's not really required for nginx - this is where the prepare step comes in. Edit molecule/default/prepare.yml:

---
- name: Prepare
  hosts: all
  gather_facts: false
  tasks:
    - name: Install netstat
      apt:
        name: net-tools
        state: installed

Running molecule test again, all of the tests pass.
One thing to be aware of here is that prepare happens before the role is run. This means it's possible to accidentally hide real role dependencies from yourself. Another option would be to install net-tools in the side-effect action, but that does not seem idiomatic to me (also, side-effects are noted to be an unstable interface in the docs). Yet another option would be to add a task to molecule/default/playbook.yml, which might be the best option.

But I already wrote my role!

It's pretty easy to add molecule testing to an existing role. From inside your role's directory, run molecule init scenario -r role_name then start writing your tests.

Final Thoughts

Molecule greatly simplifies the testing setup for ansible roles, leaving little excuse to not test roles going forward.

As I mentioned before, the Docker driver makes it quick and easy to get started, but I quickly ran into some limitations, specifically Docker doesn't work well with systemd. To test running services under a service manager, it's helpful to use one of the full VM drivers or lxd.

Bengerman

Search This Blog