While migrating and upgrading an old install of Jenkins over to version 2 the topic of adding some new views came up in conversation and the quite shiny Jenkins CI Build Monitor Plugin came up as a pretty, and quick to deploy, option. Using some canned test jobs we did a manual deploy of the plugin, configured a view on our testing machine, and I have to say it looks as good, and as easily readable from a few desks away, as we’d hoped. Read on →


As you add more jobs to Jenkins you’ll often want to start breaking them out in to smaller, more logically grouped, views. While the UI itself makes this simple it’s a manual task, and as automation loving admins we can do better than clicking around. In this post we’ll take a brief look at the jenkins-view-builder and see if it can make our lives any easier. My test case will be a simple Jenkins view that should include any jobs whose names match the test-puppet-.*-function pattern. Read on →

While there are many ways to test your code under Docker, for example puppet modules with dockunit, discussions about how to run acceptance checks against docker image and container creation are less common. In this post we’ll present one approach using the docker api and serverspec to test the creation and execution of a dockerised Redis. As our first step we’ll create the directory we’ll be testing under and a basic Dockerfile. Read on →

Continuing my journey through infrastructure testing tools we next visit testinfra, a serverspec equivalent written in python. For continuity purposes we’ll redo the Redis tests from the previous blog post. First we’ll configure a testinfra virtualenv we can use for our experiments. $ virtualenv testinfra-py-redis New python executable in testinfra-py-redis/bin/python2 $ cd testinfra-py-redis $ source bin/activate (testinfra-py-redis)[dwilson@home testinfra-py-redis]$ $ pip install testinfra # prove it works $ testinfra --version Now we have a working install of testinfra we’ll write some tests for redis. Read on →

I’m a big fan of serverspec but there are times the ruby tool chain behind it can be an annoyance and result in lots of baggage being installed. This isn’t a major problem on development machines, where many of the gems will already exist, but on production hosts the runtime dependencies can be comparatively heavy. To avoid this I’ve started looking at possible alternatives and one young, but promising, project is Goss - the tool for ‘Quick and Easy server validation’. Read on →


Constructing a large, multiple application, virtual datacenter with CloudFormation can quickly lead to a sprawl of different stacks. The desire to split things sensibly, delegate control of separate tiers and loosely couple as many components as possible can lead to a large number of stacks, lots of which need values from stacks created earlier in the run order. While it’s possible to do this with the native AWS CloudFormation command line tools, or even some clever bash (or Cumulus), having a strong, higher level tool can make life a lot easier and reproducible. Read on →

Working with multiple, related CloudFormation stacks can become quite taxing if you only use the native AWS command line tools. Commands start off gently - cfn-create-stack dwilson-megavpc-sns-emails --parameters "AutoScaleSNSTopic=testy@example.org" \ --template-file location/sns-email-topic.json but they quickly become painful. The two commands below each create stacks that depend on values from resources that have been defined in a previous stack. You can spot these values by their unfriendly appearance, such as ‘rtb-9n0tr34lac55’ and ‘subnet-e4n0tr34la’. Read on →

Once we started extracting applications into different logical CloudFormation stacks and physical templates, we began to notice quite a lot of duplication in our json when it came to declaring IAM rules. Some of our projects store their puppet, hiera and rpm files in restricted S3 buckets so allowing stacks access to them based upon environment, region, stack name and other criteria quickly becomes quite long-winded. After looking at a couple of dozen application templates and finding that over 30% of the json was IAM based it was time to find a different approach. Read on →

One of the nice little conveniences I’ve started to use in my daily work with Amazon Webservices CloudFormation is the Guard::CloudFormation ruby gem. The Guard gem “is a command line tool to easily handle events on file system modifications” which, simply put, means “run a command when a file changes”. While I’ve used a number of different little tools to do this in the past, Guard presents a promising base to build more specific test executors on so I’ve started to integrate it in to more aspects of my work flow. Read on →

One of the biggest surprises of Config Management Camp 2014 for me was how interesting Canonicals orchestration management tool, Juju has become. Although I much preferred the name ‘Ensemble’. I attended the Juju session in an attempt to keep myself out of the Puppet room and was pleasantly surprised at how much Juju had progressed since I last looked at it. Rather than being another config management solution it allows you to model your systems using “charms”, which can be implemented using anything from a bash script to a set of chef/puppet cookbooks/modules, and instead focuses on ensuring that they run across your fleet in a predictable way while enforcing dependencies, even over multiple tiers, no matter how many tools you choose to use underneath. Read on →


A while ago @ripienaar and I had a chat in a pub about monitoring, event systems and lots of related subjects. As we all know he’s way more productive than is fair and so while he’s been doing a BUNDLE of work with on subjects like monitoring frameworks and event correlation I’ve been doing some thinking (and no actual coding) about event auditing, continuous compliance and security event management. Now I’ve finished the $TIMESINK_PROJECT I’m soon going to actually need some of this stuff so I’ve started putting together a prototype framework that I’m calling DSAC - Dump Send and Correlate. Read on →

I’ve been doing a little tinkering with pre/post release checklists and compliance reporting using cucumber and some Nagios wrapping (among other things) in my test lab and recently needed to do some higher level entire environment checks before moving on to the next step. While it’s possible to wrap something like nmaps ping check and then Nagios each target it does feel like stepping back a few years in the tool chain. Read on →


Cronjobs are one of those necessary evils of any decent sized Unix setup, they provide often essential pieces of a sites data flows but are often treated as second class citizens. While I’ve already mentioned my Cron commandments I’m always looking for improvements in the rest of my cron tool set and, with Vladimir Vuksan’s cronologger, I may have found another piece of the puzzle. The concept is simple, you add a command to the front of your crontabs and it invokes your actual cron command. Read on →

The last time we interviewed for Java developers (a couple of jobs ago) it came as quite a surprise at how few of them could function without their IDE of choice. A high percentage of the candidates struggled to compile using javac, had problems navigating the docs and made a large number of very simple syntax errors that they were obviously used to their editor dealing with. At the time the more unix focused team, most of who were very long term vim and emacs users, had a number of discussions about how this should impact our rating of the candidates. Read on →