Many years ago, in the first dotcom boom, I worked for a website performance monitoring company. I was one of the early employees (developer number 3 and sysadmin number 2) and I remember being in a meeting with the company CEO who was telling us about a new pitch we were doing for $SUPERMARKET, they were going to try this new idea of shopping online and then delivering it to your door. Read on →

I’m a fan of documentation, over the years I’ve ended up supporting more than one business critical system that has less documentation than you get from a cat /dev/null. The only downside, and I’ve been bitten by a couple of things like this over the last week is the case of the spreadsheet vs the post-it note - if you have a lovely, well formatted and information dense spreadsheet that says “A is 1” and when you get to the server room the switch has a post-it, in bad scrawl, that says “B is 2” which do you believe? Read on →

After an embarrassing tale of misunderstanding, wrong locations and blind luck I recently ended up at the Introduction to data processing with Hadoop and Pig talk over at SkillsMatter - and it was excellent. For those that don’t know about Hadoop, it’s an OpenSource Java framework for data-intensive distributed applications. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google’s MapReduce and Google File System (GFS) papers. Read on →

First a disclaimer, I’m not a heavy Ruby or Java guy. Most of my coding for the last couple of years has been perl and shell - because I write little things that I need right now and those two languages excel at that (CPAN is still THE decision clincher). I recently became involved in a side project that is written in Ruby and Java though and in an excellent timing coincidence a friend returned my previously unread copy of the JRuby Cookbook. Read on →

It’s very easy to become quite blase or even cynical about new technologies but sometimes a project grabs your attention and coaxes out a “that’s very cool”, the real time augmented Google Earth had that effect on me. How long will it be before you can roll back an overlay by X weeks and see what happened in that game last Thursday or check the traffic on your new route at 7am on every Friday for a couple of weeks? Read on →

I’ve never really liked make files, I don’t think I’ve ever had to write enough C to really appreciate (or just tolerate) them, so I was a little dismissive of Rake - and I was mostly wrong. Now we’re adding a new member to the systems team I’ve been doing a lot of thinking about our tool chain - what knowledge assumptions it makes, which parts are still more manual than I’d like and where the tool chain has gaps (this is the most annoying one for me) and rake seemed like a potential addition to encode some of that process knowledge in to a tool. Read on →

Despite the fact a large percentage of the DJUGL meetups have occurred in the building I work in I’ve been very lax in attending one, and it’s been my loss. The crowd was friendly, the pizza and diet coke plentiful and the speakers enjoyable, and I’ve got every intension of making the next meeting - especially if it’s in the same building. Gareth Rushgrove started the talks with a subject very dear to my heart, deployment. Read on →

I’ve been a user of Puppet for about three years now and while on a recent dig in to some of my older classes it was a little embarrassing to see lots of file types used like this: file { '/srv/whi/maps': ensure => present, source => "puppet://$servername/whi/maps.conf", owner => 'whi', group => 'whi', mode => 0644, } file { '/srv/whi/elocs': ensure => present, source => "puppet://$servername/whi/eloc.conf", owner => 'whi', group => 'whi', mode => 0644, } Luckily as we get more experienced with a tool we can often go back and improve on the first steps. Read on →

A couple of days ago I had the chance to attend a talk on PAM and AppArmor at Skills Matter. To be honest it wasn’t what I expected, the subject level was very beginner focused, PAM only received scant coverage and the other tools were all old hands like a port scan with nmap or basic IP Tables rules. The evenings highlight for me was the coverage of AppArmor, both because it’s a very neat tech that seems orders of magnitude easier to use then SELinux and secondly because the last time I saw it mentioned was when Crispin Cowan spoke at GLLUG. Read on →

While Puppet can be used to manage large, complex environments it’s also a useful tool at the lower end of the spectrum. Using just the puppet executable and a small inline class or two you can write very useful manifests in only a handful of lines. class build_host { package { 'build-essential': ensure => installed } package { 'subversion': ensure => installed } file { "/home/dwilson/repos/": ensure => directory, owner => dwilson, group => dwilson, }}node default { include build_host} To invoke the class you just run puppet -v build-host. Read on →