As 2018 rolls along the time has come to rebuild parts of my homelab again. This time I’m looking at my monitoring and metrics setup, which is based on sensu and graphite, and planning some experiments and evaluations using Prometheus. In this post I’ll show how I’m setting up my tests and provide the Prometheus experiments with docker-compose source code in case it makes your own experiments a little easier to run. Read on →

It’s time for a little 2017 navel gazing. Prepare for a little self-congratulation and a touch of gushing. You’ve been warned. In general my 2017 was a decent one in terms of tech. I was fortunate to be presented a number of opportunities to get involved in projects and chat to people that I’m immensely thankful for and I’m going to mention some of them here to remind myself how lucky you can be. Read on →

As your terraform code grows in both size and complexity you should invest in tests and other ways to ensure everything is doing exactly what you intended. Although there are existing ways to exercise parts of your code I think Terraform is currently missing an important part of testing functionality, and I hope by the end of this post you’ll agree. I want puppet catalog compile testing in terraform Our current terraform testing process looks a lot like this: precommit hooks to ensure the code is formatted and valid before it’s checked in run terraform plan and apply to ensure the code actually works execute a sparse collection of AWSSpec / InSpec tests against the created resources Visually check the AWS Console to ensure everything “looks correct” We ensure the code is all syntactically validate (and pretty) before it’s checked in. Read on →

While trying to add additional performance annotations to one of my side projects I recently stumbled over the exceptionally promising Server-Timing HTTP header and specification. It’s a simple way to add semi-structured values describing aspects of the response generation and how long they each took. These can then be processed and displayed in your normal web development tools. In this post I’ll show a simplified example, using Flask, to add timings to a single page response and display them using Google Chrome developer tools. Read on →

Like most people I have too many credentials in my life. Passwords, passphrases and key files seem to grow in number almost without bound. So, in an act of laziness, I decided to try and remove one of them. In this case it’s my AWS EC2 SSH key and instead reuse my GitHub public key when setting up my base AWS infrastructure. Once you start using EC2 on Amazon Web Services you’ll need to create, or supply an existing, SSH key pair to allow you to log in to the Linux hosts. Read on →

I’ve been a fan of Yelps pre-commit git hook manager ever since I started using it to Prevent AWS credential leaks. After a recent near miss involving a push to master I decided to take another look and see if it could provide a safety net that would only allow commits on non-master branches. It turns out it can, and it’s actually quite simple to enable if you follow the instructions below. Read on →

A few months ago while stunningly bored I decided, in a massive fit of hubris, that I was going to write and publish a technical book. I wrote a pile of notes and todo items and after a good nights sleep decided it’d be a lot more work than I had time for. So I decided to repurpose Puppet CookBook and try going through the publication process with that instead. But (disclaimer) with a different title as there is already an excellent real book called Puppet Cookbook that goes in to a lot more depth than my site does. Read on →

With the exception of children, puppies and medical compliance frameworks managing one of something is normally much easier than managing a lot of them. If you have a lot of puppet modules, and you’ll eventually always have a lot of puppet modules, you’ll get bitten by this and find yourself spending as much time managing supporting functionality as the puppet code itself. Luckily you’re not the first person to have a horde of puppet modules that share a lot of common scaffolding. Read on →

I sift through a surprising amount, to me at least, of curricula vitae / resumes each month and one pattern I’ve started to notice is the ‘fork only’ GitHub profile. There’s been a lot written over the last few years about using your GitHub profile as an integral part of your job application. Some in favour, some very much not. While each side has valid points when recruiting I like to have all the information I can to hand, so if you include a link to your profile I will probably have a rummage around. Read on →

Have you ever noticed in the AWS console, when new instances are created, the “Tags” tab doesn’t have any content for the first few seconds? A second or two before values are added may not seem like much but it can lead to elusive provisioning issues, especially if you’re autoscaling and have easily blamed network dependencies in your user data scripts. A lot of people use Tag values in their user data scripts to help ‘inflate’ AMIs and defer some configuration, such as which config management classes to apply, to run time when the instance is started, rather than embedding them at build time when the AMI itself is created. Read on →