Ever since I started using distributed version control systems one of my fears has been accidentally publishing security credentials that could be maliciously used against me. Add to this services like AWS where you can run up a large bill very quickly, and an employer that tries to open source everything possible, and eventually you know you’re going to slip up and expose something you shouldn’t. In the past I’ve cobbled together git commit hooks to provide a basic safety net but I’m looking for a more designed solution and pre-commit is heavily in the running. Read on →

Continuing my journey through infrastructure testing tools we next visit testinfra, a serverspec equivalent written in python. For continuity purposes we’ll redo the Redis tests from the previous blog post. First we’ll configure a testinfra virtualenv we can use for our experiments. $ virtualenv testinfra-py-redis New python executable in testinfra-py-redis/bin/python2 $ cd testinfra-py-redis $ source bin/activate (testinfra-py-redis)[dwilson@home testinfra-py-redis]$ $ pip install testinfra # prove it works $ testinfra --version Now we have a working install of testinfra we’ll write some tests for redis. Read on →

As part of re-doing my home infrastructure I’m looking to add a remote location for my private git repos. My use case is a simple one, I need lots of low cost, tiny, private repos, each with a few dozen files at most. I don’t need a comprehensive set of collaboration features as I’m normally the only one working on them. My current practice is to keep my private repos on a local git server and anything open source goes to GitHub. Read on →

As possibly the last AWS using sysadmin in London who’s not invested in Terraform I’ve decided it’s time to take my quarterly look at the tool. This time around I decided to start with a basic IAM admin user and group. For my stripped down example I’m going to create a user and group, add the user to a group and set an explicit IAM group policy. As a novice terraform user I find the code easy to read, the online documentation was short but helpful and the getting started guide did indeed guide my starting. Read on →

I’m a big fan of serverspec but there are times the ruby tool chain behind it can be an annoyance and result in lots of baggage being installed. This isn’t a major problem on development machines, where many of the gems will already exist, but on production hosts the runtime dependencies can be comparatively heavy. To avoid this I’ve started looking at possible alternatives and one young, but promising, project is Goss - the tool for ‘Quick and Easy server validation’. Read on →

I’ve recently begun to look at replacing as much of my custom puppet tooling as possible with third-party, open source, code. As part of this I’m planning to update my old libvirt testing infrastructure with more modern tools, and this seems to be leading me heavily down the docker path. One of the simpler, but less known, solutions in this space seems to be Dockunit, which bills itself as “Containerized unit testing across any platform and programming language” and is remarkably simple to get started with. Read on →

After a previous comment about “ templating CloudFormation JSON from a tool higher up in your stack“ I had a couple of queries about how I’m doing this. In this post I’ll show a small example that explains the work flow. We’re going to create a small CloudFormation template, with a single Jinja2 embedded directive, and call it from an example playbook. This template creates an S3 bucket resource and dynamically sets the “DeletionPolicy” attribute based on a value in the playbook. Read on →

You can create some high value resources using CloudFormation that you’d like to ensure exist even after a stack has been removed. Imagine being the admin to accidently delete the wrong stack and having to watch as your RDS master, and all your prod data, slowly vanishes in to the void of AWS reclaimed volumes. Luckily AWS provides a way to reduce this risk, the DeletionPolicy Attribute. By specifying this on a resource you can ensure that if your stack is deleted then certain resources survive and function as usual. Read on →

While AWS CloudFormation is one of the best ways to ensure your AWS environments are reproducible it can also be a bit of an awkward beast to use. Here are a couple of simple time saving tips for refining your CFN template parameters. The first one is also the simplest, always define at the least a MinLength property on your parameters and ideally an AllowedValues or AllowedPattern. This ensures that your stack will fail early if no value is provided. Read on →

Have you ever needed to access Ansible facts from inside Puppet? well, if you ever need to, you can use the basic ansible_facts custom fact. # make sure you have ansible installed $ sudo puppet resource package ansible ensure=present # clone my experimental puppet fact repo $ git clone https://github.com/deanwilson/unixdaemon-puppet_facts.git # Try running the fact $ ruby -I unixdaemon-puppet_facts/lib/ `which facter` ansible_facts -j { "ansible_facts": { "ansible_architecture": "x86_64", "ansible_bios_date": "04/25/2013", "ansible_bios_version": "RKPPT%$DSFH.86A.0457.20DF3.0425.1251", "ansible_cmdline": { ... Read on →