Puppet has always supported templating via ERB and while it’s a powerful, flexible templating engine the ability to use any arbitrary ruby code inside a template that’s run on the puppet master sometimes raises some eyebrows. As part of a security architecture review the concept of replacing the templating engine with something that still allows looping and text manipulation without allowing too much else was discussed and led to the idea of allowing templates to be written in Liquid. Read on →

While doing some experiments with Ansible I came across a little snippet of code that I really liked - - name: manage /etc/sudoers template: src=sudoers.j2 dest=/etc/sudoers validate='visudo -cf %s' Ansible runs the command specified by ‘validate’ against the expanded templates contents and only copies the newly generated file in to place if it’s valid. This is a wonderful feature that will help stop you from making some potentially time consuming errors. Read on →

Facter 1.7 introduced support for external facts, and I gave some external fact examples, but it left a couple of small issues unresolved. One of the larger ones is the subject of syncing the external facts down to the clients. At the moment most people are managing the external facts as file resources which creates one important difference between an internal and external fact. Internal facts are synced down at the start of the run and so are available to the puppet agent within a single run. Read on →

When writing puppet modules sometimes you need to ensure that certain classes are only used within your module itself. For example a class that implements functionality based on the local operating system that should only be used by your public class. While reading though the new puppetlabs-mcollective modules source I came across a new pattern I’d not seen used in puppet before that achieves this in a very elegant way and i thought it was worth a second look. Read on →

We’re currently moving some of our early stage dev prototypes to a more automated environment and as part of this work I’m converting command line AWS resource creation to parameterised CloudFormation templates that we can use to either run multiple stacks side by side or recreate the entire stack from development to production. It’s been quite a frustrating afternoon due to some tool chain related yak shaving and some nuances in how CloudFormation works. Read on →

The most interesting links in my twitter feed this week have all been about VMWorld 2013 and the new technologies VMWare is making available in vSphere 5.5. As I don’t actually use VMWare at work (we’re using KVM and AWS at the moment) very little of it is immediately relevant to me - but then I spotted the ‘VMWare Certified Associate - Data Center Virtualization’ announcement. There is some combination of both being stuck inside waiting for family to arrive and pure envy about all the new cool stuff that tempted me in to watching the (free) VMWare Data Center Virtualization Fundamentals training course, reading the exam blue print and making an attempt at the exam. Read on →

Disclaimer: I read the early release version of this book. Some details may change between the version I read the final, published one. The first two chapters of Heroku: Up and Running are my favourite of the book. After the usual selling and explanation of virtual servers and the arrival of ‘The Cloud’ the authors cover the concept, culture and technical architecture of Heroku. The technical details of a closed company are never covered as deeply as you’d like but the material here is interesting, explains quite a lot of Herokus main technology and approaches and how th platform has evolved to enhance an opinionated work flow that seems to suit many developers. Read on →

It’s been quite a while since I’ve been heavily involved in large scale A/B testing frameworks and after reading Bandit Algorithms for Website Optimization I feel like I’ve been shown a whole new toolkit in under 90 pages. This slim book quickly explains the basic principles of A/B testing and then introduces both one of the major problems with it, sending traffic to an inefficient option, and presents three algorithms based on studies of the multi-armed bandit problem that will introduce you to new, faster, ways to determine which trials are worth investigation and further exploration. Read on →

I’ve recently had the need to create a handful of small file based providers in puppet and while trundling uphill against the ParsedFile provider I decided to have a look at how custom providers are written using the Puppet augeas providers. My sample provider is a simple one, it manages entries in /etc/shells and was quite quick to write. While augeasproviders assumes you know how Puppet types and providers are structured it does present a very standardised way of using the augeas library in order to manipulate your target. Read on →

AWS CloudFormation gives developers and systems administrators an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion. – AWS CloudFormation Homepage I’ve gone from never having used Amazon CloudFormation to building multi- tier, cross region, many availability zone deployments in a couple of months and while digging through official documentation, support requests, blog posts and sample templates I’ve put together what I’ve come to view as the ‘Four Stages of CloudFormation’. Read on →