As most of the UK Linux magazines have become very desktop focused it's nice to see some actual low-level code - packaging for OpenBSD, writing sound drivers for your NetBSD NSLU2, custom Jabber components and basic GDB were all in the two issues I skimmed. While it's not the dearly departed Sysadmin Magazine, and it could do with an editor or two - much as I could, it is a decent read and I'm considering a subscription.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /magazines Thu, 11 Feb 2010 22:11:00 GMT bsd_magazine_from_fosdem http://blog.unixdaemon.net/cgi-bin/blosxom.pl/geekstuff/grocery_arrival_excitement.html

The worst part of it was that they didn't just want monitoring, they wanted a full transaction engine (with some basic OCR), a product I can probably get away with confessing that we didn't have at the time of the sales pitch. We all knew the deal, if we didn't get it life was going to be very hard there for the next six months, so we all knuckled under. The road was long, difficult and uphill in the snow in both directions but eventually we got to the day of the pitch. Which we aced in an astounding display of luck - the new app sometimes got itself in to a little bit of a state if their website had a failure - which it did about 20% of the time. They loved the demo and wanted us to give them full coverage while they did maintenance work. If we pulled it off then we'd pretty much get the deal, none of our competition at the time could match the features, it was just the uptime that was a little worrying.

So we went out and bought a dozen small desktops, monitors and networking kit, installed them all in our spare store room, put some tables and chairs in and had a company meeting. The management were completely open about what was happening, they took questions and then asked how far we'd go to help. We covered the whole weekend from Friday night to Monday morning. Nearly the entire company chipped in, from three letter titles to sales to dev to systems to HR. We had eyes on the machines over the whole period, including when the Solaris admin, the only person to let us down, didn't make his time slot. Out of all the transactions the worst was beans, they had a new version of the code on some of the servers and it'd return very odd results for beans and break the transaction runner in horrible ways. I'll never forget the 4am calls asking what we do when they offer you a lawn-mower instead.

I placed my first ever order online with the $SUPERMARKET yesterday and hopefully it should arrive in the next couple of hours. The interface may have changed and so many of its users take the service for granted that it's a little humbling to realise how much the Internet's changed so very many things. I guess this post's about a combination of things, the best job I ever had (the company was sold in the end to one of it's competitors. I left happy in the knowledge that we ate their lunch until they gave up trying to compete and bought us), how dedicated staff can be in the right environment, why you should push the boundaries of your industry and how sometimes even cans of beans can be exciting.

I had to put a single can in the order to complete the circle. Here's to hoping they don't charge me for a lawn-mower.

Update: They didn't deliver on the night, there was a "problem with the payment" so they took the money out, using the same details and delivered it two nights later. I'll class this one as a draw.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /geekstuff Wed, 13 Jan 2010 20:05:00 GMT grocery_arrival_excitement http://blog.unixdaemon.net/cgi-bin/blosxom.pl/sysadmin/spreadsheets_vs_postitnotes.html cat /dev/null.

The only downside, and I've been bitten by a couple of things like this over the last week is the case of the spreadsheet vs the post-it note - if you have a lovely, well formatted and information dense spreadsheet that says "A is 1" and when you get to the server room the switch has a post-it, in bad scrawl, that says "B is 2" which do you believe?

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /sysadmin Tue, 12 Jan 2010 23:32:00 GMT spreadsheets_vs_postitnotes http://blog.unixdaemon.net/cgi-bin/blosxom.pl/events/hadoop-2009-talk.html Introduction to data processing with Hadoop and Pig talk over at SkillsMatter - and it was excellent.

For those that don't know about Hadoop, it's an OpenSource Java framework for data-intensive distributed applications. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google's MapReduce and Google File System (GFS) papers. I was aware of the basics but even in an hour I learned enough to know where to look for more details. Pig on the other hand is (to me) like SQL but for Hadoop, it's a lot easier to use than writing your own Java apps and simpler (and actually possible) for non-developers to read than the reams of classes required for custom jobs.

The speaker was excellent, the presentation was well timed, fluid, concise, paced just the way I like it and other than the question session the evening was very enjoyable. You can find the Hadoop slides online.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /events Sat, 17 Oct 2009 19:17:00 GMT hadoop-2009-talk http://blog.unixdaemon.net/cgi-bin/blosxom.pl/books/jruby_cookbook_short_review.html

I recently became involved in a side project that is written in Ruby and Java though and in an excellent timing coincidence a friend returned my previously unread copy of the JRuby Cookbook. The book isn't an introduction to either Java or Ruby (there are already excellent online and dead tree resources for that) but it shows where the two can meet and how to get started at those points. It's not really a book to read back to front but it is a good approach for a cookbook.

If you're curious as to how dynamic languages on static language VMs can complement each other this is a good book to flick through. Score - 6/10 - it's not the book for me right now but it does show a lot of entry points I'll probably come back to later.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /books Sat, 17 Oct 2009 19:08:00 GMT jruby_cookbook_short_review http://blog.unixdaemon.net/cgi-bin/blosxom.pl/geekstuff/dynamic_motion_on_google_earth.html real time augmented Google Earth had that effect on me.

How long will it be before you can roll back an overlay by X weeks and see what happened in that game last Thursday or check the traffic on your new route at 7am on every Friday for a couple of weeks?

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /geekstuff Wed, 30 Sep 2009 21:57:00 GMT dynamic_motion_on_google_earth http://blog.unixdaemon.net/cgi-bin/blosxom.pl/tools/commandline/raking_the_lawn.html Rake - and I was mostly wrong.

Now we're adding a new member to the systems team I've been doing a lot of thinking about our tool chain - what knowledge assumptions it makes, which parts are still more manual than I'd like and where the tool chain has gaps (this is the most annoying one for me) and rake seemed like a potential addition to encode some of that process knowledge in to a tool. I've only added little rakefiles here and there but they do make certain tasks nicer (plus I like the inline descs).

I've not yet worked out any general rules for when to use a shell script and when to use rake but if nothing else it's helping me spend some time on my ruby skills. The best rake starting points I found were Martin Fowlers rake article and the rake release notes.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /tools/commandline Wed, 30 Sep 2009 21:48:00 GMT raking_the_lawn http://blog.unixdaemon.net/cgi-bin/blosxom.pl/events/djugl_200909.html DJUGL meetups have occurred in the building I work in I've been very lax in attending one, and it's been my loss.

The crowd was friendly, the pizza and diet coke plentiful and the speakers enjoyable, and I've got every intension of making the next meeting - especially if it's in the same building.

Gareth Rushgrove started the talks with a subject very dear to my heart, deployment. The talk went quite quickly with an intro to Fabric and a live demo of a deployment. The demo went as most demos do but fabric itself looks interesting, especially when you consider that our websites are written in Python. Unfortunately the two different versions (with different goals) and the fact that a new owner's pushing the development now means it's not something I can drop in right now with any degree of comfort. I'd like other people to find the rough edges first so for now I'll stick with the plan of getting Capistrano involve id in one of our more self contained projects alongside puppet.

Although I disagreed with little bits here and there it's always nice to hear a developers point of view on this stuff.

Ben Firshman was up next with a selection of talks, the Celery distributed task queue (very nice API but passes pickled python objects so only good for Python at each end projects), a rewrite of MPTT (a topic of which I know nothing) and some highlights about py.test (a quite nice Python test framework). I've been lucky enough to work with Ben over the last few months and he's someone to keep an eye on technically.

This was the last DJUGL organised by Robert Lofthouse, he's passed the baton on to Gareth, and from the people I met at the event, and he can pass it on proud of the group he's put together.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /events Sun, 27 Sep 2009 18:53:00 GMT djugl_200909 http://blog.unixdaemon.net/cgi-bin/blosxom.pl/tools/puppet/simplifying_file_permissions_in_puppet_manifests.html Puppet for about three years now and while on a recent dig in to some of my older classes it was a little embarrassing to see lots of file types used like this:

  file { "/srv/whi/maps":
    ensure => present,
    source => "puppet://$servername/whi/maps.conf"
    owner  => whi,
    group  => whi,
    mode   => 644
  }
  file { "/srv/whi/elocs":
    ensure => present,
    source => "puppet://$servername/whi/eloc.conf"
    owner  => whi,
    group  => whi,
    mode   => 644
  }

Luckily as we get more experienced with a tool we can often go back and improve on the first steps. By using an explicit File { settings } inside a class you can assign a sensible set of defaults to all the instances of the same type that lack overriding settings. So we can shorten the previous example to -

File {
  owner => whi,
  group => whi,
  mode  => 644
}

file { "/srv/whi/maps":
  ensure => present,
  source => "puppet://$servername/whi/maps.conf"
}

file { "/srv/whi/elocs":
  ensure => present,
  source => "puppet://$servername/whi/eloc.conf"
}

While this isn't a huge win in raw characters typed (although in longer manifests they start to mount up) it does move all the common settings in to a single location (keeping us clear of DRY violations) and it leaves only the differences between file type definitions.

You can also apply those kind of settings (such as Exec { path => "path:list" } at the server level by including them in a top level file and then overriding them as needed in each module. If you do this then you need to be aware that any declared type that doesn't override it gets the global setting, which can lead to the odd action from afar head scratching.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /tools/puppet Wed, 23 Sep 2009 22:16:00 GMT simplifying_file_permissions_in_puppet_manifests http://blog.unixdaemon.net/cgi-bin/blosxom.pl/events/ubuntu_security_talk_2009_09.html

The evenings highlight for me was the coverage of AppArmor, both because it's a very neat tech that seems orders of magnitude easier to use then SELinux and secondly because the last time I saw it mentioned was when Crispin Cowan spoke at GLLUG. It's great to see it in a mainline distro and I've added it back on to my experiment with list.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /events Thu, 17 Sep 2009 22:06:00 GMT ubuntu_security_talk_2009_09 http://blog.unixdaemon.net/cgi-bin/blosxom.pl/tools/puppet/stand_alone_puppet.html Puppet can be used to manage large, complex environments it's also a useful tool at the lower end of the spectrum. Using just the puppet executable and a small inline class or two you can write very useful manifests in only a handful of lines.

class build-host {
  package { "build-essential": ensure => installed }
  package { "subversion":      ensure => installed }

  file { "/home/dwilson/repos/":
    ensure => directory,
    owner  => dwilson,
    group  => dwilson,
  }
}

node default {
  include build-host
}

To invoke the class you just run puppet -v build-host.pp. It's also worth pointing out the node name of default. This saves you manually changing the manifest whenever you move to another machine. While it wouldn't be hard to replace the above example with a shell script, by using puppet you can easily access the built-in abstractions (which package manager to use, how should you add users) and remove a lot of scaffolding code. And then when you're done you can promote the class to your managed infrastructure.

I've used this to bootstrap provisioning servers (why should the provisioning host be the only machine that wasn't provisioned?), test small but annoying new classes on scratch servers and I'm currently working on integrating it with a small subversion backup testing project in my spare time at work (so very slowly).

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /tools/puppet Thu, 17 Sep 2009 21:33:00 GMT stand_alone_puppet http://blog.unixdaemon.net/cgi-bin/blosxom.pl/books/javascript_good_parts_short_review.html JavaScript: The Good Parts is only 124 pages it took me a lot of attempts to work my way through it. A combination of the authors attitude and the dry presentation put me off within the first three chapters every time i tried to read the book.

However a side project I was helping out on needed some JavaScript reviewed and considering how little of the language I knew I forced myself to work through the book and I'm glad I did - despite its short comings it's an excellent introduction to the language for programmers with a couple of other languages under their belt. The main parts of the language, such as syntax railroad diagrams, objects, inheritance and regular expressions are covered in a very basic, but to the point style with some excellent little code snippets. The examples in this book help you think of the language itself as mutable and just another tool to bend to your needs.

Once I accepted the authors style I liked the book. It would have been better with a little less repetition and maybe even as a pocket reference but it's an excellent book on JavaScript none the less. Score - 7/10

Note - when trying the samples, and experimenting with the language in general, I used the Mozilla Rhino JavaScript interpreter. It was nice to have a non-web playground for getting to grips with the language without needing to work around the idiosyncrasies that come with web browsers.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /books Mon, 14 Sep 2009 22:25:00 GMT javascript_good_parts_short_review http://blog.unixdaemon.net/cgi-bin/blosxom.pl/books/land_the_tech_job_you_love_short_review.html Land The Tech Job You Love covers all these critical points and expands the other parts of the job seeking process - researching the company, preparing for the interview and how to answer the more ambiguous questions that often come up.

The book is well written and has lots of action points that can help you along the way. I don't think I'd follow all the advice as given, some of it seems very American, but the book does raise a lot of points you should at least be aware of during your job hunt.

Score - 6/10

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /books Sun, 13 Sep 2009 18:29:00 GMT land_the_tech_job_you_love_short_review http://blog.unixdaemon.net/cgi-bin/blosxom.pl/misctech/verified_by_visa_designed_by_idiots.html

If you've been blessed enough to never have it ruin your transaction here's the short version - in the middle of paying for something you get bounced, with no clue where you're going and how secure it is, to a third party site, which is completely safe as it's run by visa, that then gets you to enter a password. Or if you don't know it, create a new one using nothing more than what's on your card.

Firstly how stupid is that? What ever happened to something I know, something I have? If I find a lost card I can reset the Verified by Visa password using nothing more than my powers of reading and typing. While we're on the subject of passwords - you're not allowed to use special characters. Numbers and letters only. Thanks, rule out half the possibilities in one sweep for me. And what's with the remembering history? This thing makes elephants look like /tmp... it remembered all the verified passwords I've ever used going back about ten iterations - and O change the password a lot because I can't use a decent one and it's easier to reset it than to dig out the old one.

Now suppose you've logged in, got the password right and clicked next, what happens? You get a session / transaction timeout and you have to go all the way back through your order. Thanks for that.

If your site makes me jump through these fake security hoops then I'll go elsewhere. I won't play along anymore - you'll just lose my custom. And hopefully that of many other people.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /misctech Wed, 02 Sep 2009 22:07:00 GMT verified_by_visa_designed_by_idiots http://blog.unixdaemon.net/cgi-bin/blosxom.pl/operatingsystems/linux/ubuntu_developer_week.html Ubuntu Developer IRC workshops. While I don't actually use Ubuntu at work it's always a good idea to keep up with the new and shiny, and as an extra incentive a lot of the technical details mentioned also apply to Debian, which I do have to admin on a daily basis.

While the IRC logs don't go in to huge details the two sessions I've looked at (getting started and packaging perl modules) each contain enough useful links to make them worth my time.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /operatingsystems/linux Tue, 01 Sep 2009 04:23:00 GMT ubuntu_developer_week http://blog.unixdaemon.net/cgi-bin/blosxom.pl/nottech/follow_the_mockery.html

Leaving incompetent people in place destroys morale for the good staff and encourages them to look elsewhere. When there's a bad apple or two lowering the teams value, and the quality of their output, pure professional pride only gets you so far. At some point you need to have a culture that rewards effort and punishes wastes of space.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /nottech Mon, 31 Aug 2009 20:58:00 GMT follow_the_mockery http://blog.unixdaemon.net/cgi-bin/blosxom.pl/books/cloud_application_architectures.html Cloud Application Architectures isn't a bad place to start.

The book is a slim tome, it's easy to read in a couple of sittings and covers all the basics. The author felt more than a little biased towards the cloud (IMHO) but what do you expect from someone leading the push? The book is well written, clear in making its points and the worst omission / error are a couple of missing figures, but the text works fine without them.

I'd be remiss if I didn't point out the Berkley Cloud paper (PDF), a great paper for defining terms regarding the cloud, and any of the keynotes by Simon Wardley (especially the one from FrOScon 2009) as other good starting points - both of which are free.

If you're interested in how the cloud could play a part in your environment and want something a little more concrete (and AWS focused) then this book is for you. 7/10.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /books Mon, 31 Aug 2009 20:26:00 GMT cloud_application_architectures http://blog.unixdaemon.net/cgi-bin/blosxom.pl/books/project_california_short_review.html Project California: a Data Center Virtualization Server ticks a lot of the boxes, which is a little misleading as half the book is about solid hardware level details that are actually rarely covered.

While this makes the first half more than a little dry it does introduce concepts that many of us take for granted, such as why DDR3 is faster than DDR2. The second half takes you through the Cisco UCS stack and where the benefits are. It's a good starting point but I'd hoped for some more meat, maybe even a case-study or two. The book answered some of my questions but it's not amazingly comprehensive so expect to do a lot more digging after you've finished reading it.

The book is self-published (Via Lulu) so it's probably worth mentioning the quality - it's the same as any other book I've bought recently. No better and no worse, which is actually pretty impressive.

6/10 - dry, fills a niche but covers a lot of general material not specific to UCS.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /books Sun, 30 Aug 2009 11:05:00 GMT project_california_short_review http://blog.unixdaemon.net/cgi-bin/blosxom.pl/events/froson_2009.html FrOSCon 2009. Along the way I experienced an airport evacuation due to fire alarm, a delayed flight, four trains (one in the wrong direction) and numerous kindly old German ladies that took pity on me and gradually got me in the right direction. And it was worth every second.

I'd never really considered going to FrOSCon before, the percentage of talks given in German is quite high and I don't speak a word of the language but this year there was an excellent line up of speakers (all presenting in English) in the OpenSQLCamp room. As I'm getting more and more MySQL requests at work I thought it'd be a handy thing to go along and learn from.

In general it reminded me of the earlier FOSDEMs in tone, especially with the selection of project based developer rooms. The only annoyance was that so many sysadmin related talks, with excellent attention grabbing titles, were in German only. I plan on working my way through the slides in the hope that I can get at least a basic idea of their subject matter.

A large percentage of the sessions were recorded and I'm looking forward to catching up on the sessions where I was already booked or engaging in the hallway track. The only annoyance was that so many sysadmin related talks, with excellent titles, were in German only. I plan on working my way through the slides in the hope that I can get at least a basic idea of their subject matter.

Speaker highlights for me were Dag Wieers and Simon Wardley, both are very entertaining, highly practised speakers that grab the audience and don't let go. I especially like watching the audience in swardleys cloud computing talks as they gradually come to see how it's going to impact all of us eventually. If he was paired with someone in the mobile space with the same kind of eloquence we'd have a nice road map of the tech future.

Although the English language talks were a little thin on the ground during the second day the organisers did an excellent job (although the newbies like me could have done a few pointers about things like the BBQ, do we buy tickets? Pay at the counter?) and with a little luck I'll have the money to go back again next year.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /events Sun, 30 Aug 2009 11:00:00 GMT froson_2009 http://blog.unixdaemon.net/cgi-bin/blosxom.pl/sysadmin/find_and_replace_interview_question.html

"How would you change all occurrences of 10.23.34.10 to 10.23.34.101 in a text file?

While most of the candidates failed this one on account of skipping the question we had a couple make attempts (oddly all using the vim replace syntax). While I was hoping for a little bit of sed or perl a solution's a solution. Unfortunately theirs all had edge cases.

First was the obvious trick in the question. All the answers (apart from one) came back with something like this - %s/10.23.34.10/10.23.34.101/. First up we have the missing boundary test. While that regex is fine for 10.23.34.10 it breaks quite badly on 10.23.34.101 (it replaces it with 10.23.34.1011). Secondly the dots are unescaped, which means it's not only going to work on IPs, but number sequences.

I was actually really surprised at how few of the admins could write an awk / perl / shell script and how many paid very little attention to what the regex actually matched. Still, while the question seemed easy enough it had enough awkwardness to enable further discussion about regexs, data checking and the evil of manual changes. And it showed that everybody likes vim :)

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!]]> Dean Wilson <dean.wilson@gmail.com> /sysadmin Fri, 28 Aug 2009 21:54:00 GMT find_and_replace_interview_question