A few projects ago we had a JSON app with quite a fiddly config file that was undergoing rapid iteration. Although we never deployed an invalid JSON config we hit a couple of snags with config files that didn’t quite match up to the applications expectations. A proposed solution was to produce a JSON Schema document we could use in both integration tests and to ensure the JSON we generated in Puppet was both well formed and, more importantly, valid for that version of the application. Read on →

As you add more jobs to Jenkins you’ll often want to start breaking them out in to smaller, more logically grouped, views. While the UI itself makes this simple it’s a manual task, and as automation loving admins we can do better than clicking around. In this post we’ll take a brief look at the jenkins-view-builder and see if it can make our lives any easier. My test case will be a simple Jenkins view that should include any jobs whose names match the test-puppet-.*-function pattern. Read on →

While there are many ways to test your code under Docker, for example puppet modules with dockunit, discussions about how to run acceptance checks against docker image and container creation are less common. In this post we’ll present one approach using the docker api and serverspec to test the creation and execution of a dockerised Redis. As our first step we’ll create the directory we’ll be testing under and a basic Dockerfile. Read on →

Ever since I started using distributed version control systems one of my fears has been accidentally publishing security credentials that could be maliciously used against me. Add to this services like AWS where you can run up a large bill very quickly, and an employer that tries to open source everything possible, and eventually you know you’re going to slip up and expose something you shouldn’t. In the past I’ve cobbled together git commit hooks to provide a basic safety net but I’m looking for a more designed solution and pre-commit is heavily in the running. Read on →

Continuing my journey through infrastructure testing tools we next visit testinfra, a serverspec equivalent written in python. For continuity purposes we’ll redo the Redis tests from the previous blog post. First we’ll configure a testinfra virtualenv we can use for our experiments. $ virtualenv testinfra-py-redis New python executable in testinfra-py-redis/bin/python2 $ cd testinfra-py-redis $ source bin/activate (testinfra-py-redis)[dwilson@home testinfra-py-redis]$ $ pip install testinfra # prove it works $ testinfra --version Now we have a working install of testinfra we’ll write some tests for redis. Read on →

As part of re-doing my home infrastructure I’m looking to add a remote location for my private git repos. My use case is a simple one, I need lots of low cost, tiny, private repos, each with a few dozen files at most. I don’t need a comprehensive set of collaboration features as I’m normally the only one working on them. My current practice is to keep my private repos on a local git server and anything open source goes to GitHub. Read on →

As possibly the last AWS using sysadmin in London who’s not invested in Terraform I’ve decided it’s time to take my quarterly look at the tool. This time around I decided to start with a basic IAM admin user and group. For my stripped down example I’m going to create a user and group, add the user to a group and set an explicit IAM group policy. As a novice terraform user I find the code easy to read, the online documentation was short but helpful and the getting started guide did indeed guide my starting. Read on →

I’m a big fan of serverspec but there are times the ruby tool chain behind it can be an annoyance and result in lots of baggage being installed. This isn’t a major problem on development machines, where many of the gems will already exist, but on production hosts the runtime dependencies can be comparatively heavy. To avoid this I’ve started looking at possible alternatives and one young, but promising, project is Goss - the tool for ‘Quick and Easy server validation’. Read on →

I’ve recently begun to look at replacing as much of my custom puppet tooling as possible with third-party, open source, code. As part of this I’m planning to update my old libvirt testing infrastructure with more modern tools, and this seems to be leading me heavily down the docker path. One of the simpler, but less known, solutions in this space seems to be Dockunit, which bills itself as “Containerized unit testing across any platform and programming language” and is remarkably simple to get started with. Read on →

After a previous comment about “ templating CloudFormation JSON from a tool higher up in your stack“ I had a couple of queries about how I’m doing this. In this post I’ll show a small example that explains the work flow. We’re going to create a small CloudFormation template, with a single Jinja2 embedded directive, and call it from an example playbook. This template creates an S3 bucket resource and dynamically sets the “DeletionPolicy” attribute based on a value in the playbook. Read on →