Ansible is one of the things that you just can not chose to ignore when you reach a certain level of maturity inside your it infrastructure where the "just do it" approach does not fit anymore. One of the problems with this thou is the missing test infrastructure for ansible roles. There are some tools build around ansible that make atomic testing of ansible roles easier like this which creates wrapper to test a role with its default values without the hassle of creating a test inventory and an additional playbook just for testing purposes.

Another thing is that you want to have some kind of standard platform and a way to reproduce test runs in an automated fashion.

Docker

To make the test environment reproducible, docker is one of the most easy to use solutions for this.  While other solutions that are based on minimal VMs can be better in some edge cases, i will focus on a container based solution for this but the concept will work with some kind of VM orchestrating tool like open stack or open nebula as well.

Image Preparation

Ansible roles should be as complete as possible, so the testing environment should be as minimal as possible by also being diverse in the flavor of the distribution userland. Luckily ansible does not need a daemon running but just a python environment installed. To be even more minimal we can let our tests just run with a local connection (as in server = target). Most linux distributions have a package for ansible or a package for pip.

A Dockerfile could look something like this for as many distributions as feasible:

FROM ubuntu:18.04

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
      software-properties-common \
      apt-utils \
    && apt-add-repository ppa:ansible/ansible \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
      ansible \
      python-apt \
      git-core \
      openssh-client \
    && apt-get clean

CMD ["ansible-playbook", "--help"]

Role Setup

With our Docker images now done and pushed to a private registry or the docker hub we can focus on the role setup. A ansible role folder and file layout can easily be created via ansible-galaxy

ansible-galaxy init test-role

This will already create a test folder that just wants to be filled. For my scenario we will just write a simple playbook and a test inventory to bind the execution of the play to localhost for our test containers

test-playbook.yml

---

- name: test playbook
  hosts: all
  gather_facts: True
  roles:
   - test-role  

test-inventory

[debian]
localhost ansible_connection=local ansible_python_interpreter=/usr/bin/python2

[arch]
localhost ansible_connection=local ansible_python_interpreter=/usr/bin/python2

[centos]
localhost ansible_connection=local ansible_python_interpreter=/usr/bin/python2

Drone.io

Drone.io (or just Drone) is a Container native CI/CD solution and one of my favorites to use whenever i have the possibility to work in a container environment. It supports the execution of commands in custom images as well as a wide variety of community supported plugins (docker images that work with the drone api to create additional functionality like sending a telegram message f.e.). It is a open source tool with the possibility to buy enterprise grade support or use it as a service, so if you can, support your local open source project ;-)

Drone is very easy to setup and integrates well into existing git frontends like gitea, bitbucket or github, but enough with the fanboying. Drone gets its pipeline descryption like other tools like this in form of a yml file that looks something like this:

---
kind: pipeline
name: ansible-test

platform:
  os: linux
  arch: amd64

concurrency:
  limit: 9

steps:

  - name:  prepare_workspace
    image: alpine
    commands:
      - mkdir -p /drone/src/tests/test-role
      - mv defaults /drone/src/tests/test-role/
      - mv files /drone/src/tests/test-role/
      - mv handlers /drone/src/tests/test-role/
      - mv meta /drone/src/tests/test-role/
      - mv tasks /drone/src/tests/test-role/
      - mv templates /drone/src/tests/test-role/
      - mv vars /drone/src/tests/test-role/
    when:
      event:
      - push
      - pull_request

  - name: test-debian7
    image: lerentis/ansible:debian-7
    commands:
      - ansible-playbook -i tests/inventory tests/test-playbook.yml --syntax-check
      - ansible-playbook -i tests/inventory tests/test-playbook.yml
    depends_on:
      - prepare_workspace

  - name: test-debian8
    image: lerentis/ansible:debian-8
    commands:
      - ansible-playbook -i tests/inventory tests/test-playbook.yml --syntax-check
      - ansible-playbook -i tests/inventory tests/test-playbook.yml
    depends_on:
      - prepare_workspace

  - name: test-ubuntu14
    image: lerentis/ansible:ubuntu-14.04
    commands:
      - ansible-playbook -i tests/inventory tests/test-playbook.yml --syntax-check
      - ansible-playbook -i tests/inventory tests/test-playbook.yml 
    depends_on:
      - prepare_workspace

  - name: test-ubuntu16
    image: lerentis/ansible:ubuntu-16.04
    commands:
      - ansible-playbook -i tests/inventory tests/test-playbook.yml --syntax-check
      - ansible-playbook -i tests/inventory tests/test-playbook.yml 
    depends_on:
      - prepare_workspace

  - name: test-ubuntu18
    image: lerentis/ansible:ubuntu-18.04
    commands:
      - ansible-playbook -i tests/inventory tests/test-playbook.yml --syntax-check
      - ansible-playbook -i tests/inventory tests/test-playbook.yml 
    depends_on:
      - prepare_workspace

  - name: test-arch
    image: lerentis/ansible:arch
    commands:
      - ansible-playbook -i tests/inventory tests/test-playbook.yml --syntax-check
      - ansible-playbook -i tests/inventory tests/test-playbook.yml 
    depends_on:
      - prepare_workspace

  - name: test-centos7
    image: lerentis/ansible:centos-7
    commands:
      - ansible-playbook -i tests/inventory tests/test-playbook.yml --syntax-check
      - ansible-playbook -i tests/inventory tests/test-playbook.yml 
    depends_on:
      - prepare_workspace

  - name: notify
    image: appleboy/drone-telegram
    settings:
      message: "Commit {{ commit.link }} ran with build {{ build.number }} and finished with status {{ build.status }}."
    to:
      from_secret: userid
    token:
      from_secret: bot_token
    when:
      status:
      - failure
      - success
    depends_on:
      - prepare_workspace
      - test-debian7
      - test-debian8
      - test-ubuntu14
      - test-ubuntu16
      - test-ubuntu18
      - test-arch
      - test-centos7

This yml file will now prepare the folder structure as needed for the test container to find the role that is listed in the test playbook and execute it on a lot of different linux flavours in parallel.

The Ugly

As with everything there are downsides here as well. Anything that you would do with a handler or any kind of init system will fail inside a container so these steps have to be excluded here. I chose to do this with a when clause to skip a step that i know will fail within a container

- name: reload service
  become: yes
  service:
    name: test
    state: reloaded
  when: ci_run is undefined

Conclusion

Automated testing with ansible roles is nothing that has a final and complete solution but i think to have some kind of testing structure is better than to have none at all. As with software development, ansible roles should be tested before using them even if it's just to note some syntax errors before running a playbook in production

References