Fri, 30 Jan 2009
MySQL Uni - Scalability Challenges in an InnoDB-based Replication Environment
I recently 'attended' my first MySQL University presentation - Scalability Challenges in an InnoDB-based Replication Environment.
The service itself is great, you sign up, log in and then
watch the speaker present in one window while listening to him speak and
reading the slides (in the main part of the screen). Everything you'd expect
really.
The subject wasn't anymore exciting than you'd guess (but what do you expect with that title?) but the speaker knew his stuff and a couple of the Solaris commands shown will be useful to me in the future. Most valuable for me from the time invested in watching is that between this and the Jan 09 LOSUG I'm being pushed to invest some time in OpenSolaris and potentially deploy an instance or two of it in our staging environment for performance and debugging reasons. While Linux is pushing the boundaries in many ways Solaris is still ahead when it comes to working out the why, when and what.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/30 21:19 | /presentations | Permanent link to this entry | This entry and same date
LOSUG - Jan 09 Wrap Up
I'm going to try and get to more LOSUG meetings this year and the January presentation by MC
Brown has done nothing to put me off. Although some of the audience tried
their best...
First up - the good. The actual presentation, MySQL/DTrace and Memcached, was very well done. The speaker was funny, well rehearsed and knew his material extremely well. The MySQL DTrace probes are made to be used in demos and are very enticing.
Now for the bad. The talks technical coverage was quite light on the MySQL/DTrace part and the memcache section was painful. Big parts of the audience just didn't seem to get the idea. The speaker had to constantly backtrack to keep more than a handful of us with him. I had to resist the urge to join in and ask questions like 'how do you do joins?' just to see how big the vein on his forehead would get.
Still, the speaker must have been good as I've already played with the OpenSolaris Live CD they gave away. I'm looking forward to next month.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/30 21:17 | /events | Permanent link to this entry | This entry and same date
Tinselworm DVD (Bill Bailey) - Short Review
It took me a while to warm to Bill Bailey as a comedian. His slower,
more laid back humour is a change in pace from what I normally like but
Tinselworm, like all his live shows, is an excellent mix of music and
mirth. If you like Bill Bailey then it's another must have, if you've never
watched him then go for a little trawl through youtube (start with this) and if you don't
like him then you're just weird.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/30 21:03 | /movies | Permanent link to this entry | This entry and same date
Mon, 26 Jan 2009
My CPAN Tidy - Jan 2009
It's been a while since I gave any attention to my CPAN modules but as an
incentive to get more hands on with git I added them to my own gitweb, fixed the
two that were failing tests and tided up some of the complaints from
CPANTS.
I'm sure I've missed something (or got it flat out wrong) but it's nice to have at least a local copy of my modules without any issues remaining. Until someone finds more of course... The first two updates have been sent to CPAN and I'll do the others later in the week if the new ones are declared fine.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/26 20:52 | /perl | Permanent link to this entry | This entry and same date
Thu, 22 Jan 2009
LOSUG 2009 - A MySQL Must See
LOSUG is one of
Londons best kept tech secrets. It's hosted in a nice venue, often has a very
knowledgeable audience full of Sun engineers and this month will be covering
MySQL/DTrace and Memcached.
If you're a sysadmin or a developer interested in getting more, or better, metrics and understanding of how and what your system is doing make sure you book a place.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/22 18:50 | /events | Permanent link to this entry | This entry and same date
Mon, 19 Jan 2009
My Pet Puppet Hate - Adding New Types
Now that chef is out and about people that accepted the massive improvement
over all the existing host configuration managers that is Puppet
will probably be casting a weary eye its way.
I've got a little too much in puppet at my current position to look at moving for a while yet but now the competition is rising its time to get my boot in and point out what, for me, is the worst part of puppet; how difficult it is to add new types.
One of the greatest strengths of tools like Nagios and Munin is the community tools provided. Nagios has a decent selection of plugins out of the box but a quick google or check of NagiosExchange shows dozens of additions (including some of my own Nagios Plugins.
With puppet on the other hand once you reach the point where you want to write custom types it all gets very heavy, very quickly. The biggest issue, to me at least, is that the level of abstraction feels wrong. Adding a simple type that will add a line to a config file for example (such as /etc/sysctl.conf) should be an easy task but the lack of documentation and the different approaches taken by the existing types (which seem to have been done at very different times and feel quite different) make it awkward to crib from. If instead there was a simple type where you changed the filename and the separator for example then a lot of custom types become within reach for less ruby skilled users.
On the flip-side my current hope is the Augeas type. It understands a lot of config files, provides consistent access to add and append to them and can be wrapped in defines.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/19 23:00 | /sysadmin | Permanent link to this entry | This entry and same date
Wed, 14 Jan 2009
Soon to be With Added Git?
Despite setting up my own
gitweb
install I'm still not using git regularly enough to be comfortable with it
so today I went through the Peepcode Press Git Internals book/PDF. While
the diagrams and details of what happens under the cover are useful it's
the wrong level for me as a basic user. To ease myself in to the move
from subversion for some of my personal projects I found Git Magic to be more
useful.
I know git requires a mental shift and it's a very complex and powerful tool but for my own needs I'll probably never use more than 10% of its capabilities. Unfortunately most of the projects I use and need to submit patches to have switched - so I'll be a happy sheep and go along for the ride. Even if it turns out to be a roller coaster.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/14 18:14 | /tools/commandline | Permanent link to this entry | This entry and same date
Mon, 12 Jan 2009
Prague - It's cold out there...
... but the beer is very cheap. Which I know is an important thing for my
readers. I can also agree with their choice of food, lots of pork and
goulash with stodgy dumplings and thick sauce. Pig knuckle is much nicer
than it sounds. I spent a long weekend in Prague, it was -10 for most of
it but luckily the city isn't very big and you can reach all the usual
tourist spots by foot if you've got a day or two.
Most of the actual tourist spots are a little... underwhelming but the highlight for me was a concert we attended at the Narodni Muzeum (National Museum). We walked in and were pointed to our seats; which were cushions on the staircases. Ten minutes later the Czech String Chamber Orchestra walked in and were amazing for just over an hour. It felt very private and was worth every penny (it cost just over 20 quid). On the other hand the Prague Astronomical Clock and Wenceslas Square are on every list of the cities "top 10 locations" and both were disappointing.
Oh,and just in case anyone else is as dumb as me - Prague does not use Euros... Fortunately they have cash machines in the airport lobby.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/12 22:18 | /nottech | Permanent link to this entry | This entry and same date
Penetration Testing in a Sentence
Penetration testing is tactical. It provides tangible, actionable
information -- Ivan Arce
It's been a while since I've been involved in pen testing but the above quote from Ivan is perfect and its meaning all too often overlooked. When you invest the time in something like pen testing or performance tuning you should always come away with a list of actionable tasks.
By doing this you ensure the work wasn't pointless (or if it was avoid repeating the mistake) and have something you can present to stake holders to get buy in for the next time. It's also easier to automate some of the scut work if you have a solid list of tasks and outcomes.
On the flip side it's also worth considering how actionable some of your other automated processes are. Does every Nagios error have a solution to resolve it? Do actions emerge from your graphs or do they just add background noise?
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/12 21:44 | /sysadmin | Permanent link to this entry | This entry and same date
Thu, 08 Jan 2009
Limiting Failed SSH Logins using PAM
Ever wanted to limit the number of ssh login attempts a user can make
before their account gets locked? Well, not really, but when brute force
tools are so common and easy to use it's another useful trick in the
sysadmins arsenal.
In this example I'll show you how to install, configure and audit failed ssh loging attempts on Linux. While the PAM mod_tally module is available for a number of different distros and Unix variants we'll set it up on Debian. First of all grab the package if you don't already have it -
apt-get install libpam-modulesNow we've installed mod_tally you have to add a couple of lines to the
/etc/pam.d/ssh PAM config file for ssh.
$ vi /etc/pam.d/ssh
# add before @include common-auth
# lock the buggers after 3 attempts
auth required pam_tally.so onerr=fail deny=3
# add before @include common-account
# resets count if login successful
account required pam_tally.so reset
The ordering of the lines is important. In some configurations previous PAM checks can shortcut the full process. While this post isn't the place to learn all about PAM the first line of the example sets what to do if something strange happens, such as the log file being unavailable (onerr=fail) and how many times a login can fail before being locked (deny=3). The second line tells PAM to reset the counter once a successful login has occurred for a specific user. If you leave this one out every failure is remembered for ever and eventually all will be locked out.
Now you're up and running how do you find out what's happening? If you
want to look at the current status then you can run the
pam_tally command and you'll see output like this -
root@pam:/etc/pam.d# /usr/sbin/pam_tally
User ajones (1000) has 2
User lockme (10010) has 4
pam_tally also logs wherever your authentication events go
(/var/log/auth.log on Debian by default ) so you can keep
historical information or feed the attempts in to your normal log
monitoring systems
# sample log line
Jan 8 19:16:24 pam pam_tally[30086]: user lockme (10010) tally 4, deny 3
To close the loop let's cover resetting the locked accounts. If you have
a user complaining then you can run pam_tally --reset --user
lockme to clear their tally. Another option (worth considering) is a
scheduled reset. This gives you the benefit of slowing down brute force
attacks while not requiring you to unlock all accidentally locked accounts.
The simplest way to do this is to add a cronjob that runs a reset. Newer
versions have an unlock_time=n option you can supply in the
ssh PAM config file but that didn't work for me under Etch.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/08 23:31 | /security | Permanent link to this entry | This entry and same date
Wed, 07 Jan 2009
Which Zones Have a Specified Subdomain? - DNS Delvings (1)
It's been another day of many DNS changes and while the work itself has
been amazingly dull, life draining, scut work at least one positive
thing's come out of it - my appreciation for the Net::DNS perl
module has grown.
While it's possible to do nearly anything DNS query related with the
dig command it's a lot easier to extract the data and
reuse certain fields if you have access to a decent data structure rather
than grepping bits of text out. Over the next couple of days, while I'm
elbow deep in our domain name system, I'll be posting, hopefully useful,
little snippets of code to illustrate how you can get a lot of value from
little code.
Today we have a script that accepts checks for the presence of the
specified subdomain in all the domains mentioned on the command line and
reports success or failure. I've not had to run it in anger yet but it
should, fingers crossed, save me a lot of digging around tomorrow. You
invoke the script like this - check-subdomains-presence blog
example.org example.com.
#!/usr/bin/perl -w
use strict;
use warnings;
use Net::DNS;
die "$0: please supply a subdomain to look for and one or more zones to look in\n"
unless @ARGV >= 2;
my $domain = shift;
my @zones = @ARGV;
my $res = Net::DNS::Resolver->new;
for my $zone ( @zones ) {
my $query = $res->query( "$domain.$zone", "A");
if ($query) {
foreach my $rr (grep { $_->type eq 'A' } $query->answer) {
print "Present - $domain.$zone\n";
}
} else {
warn "Absent - $domain.$zone - ", $res->errorstring, "\n";
}
}
The code is short, easy to read through and will hopefully whet your appetite for the longer posts to come...`
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/07 23:11 | /services/dns | Permanent link to this entry | This entry and same date
Tue, 06 Jan 2009
Diffing Files Over Multiple Servers - rd-differ
Adhoc changes are a very bad thing in many ways, one of the worst is how
often they are not fully implemented across all the servers or even
pulled back to staging. In an attempt to sanity check the config files when
we have to make these little hacks I oddly-proudly present - rd-differ. A tool for
diffing config files over multiple machines.
The idea is simple, you tell it the file or directory you're interested
in, specify a single machine as the baseline and then specify a number of
others as the machines to check against it. A sample invocation looks like this
rd-differ /etc/apache2 10.10.100.111 10.10.100.112
10.10.100.113 and the output is show as a diff.
The files are rsynced down using ssh so your usual keys will work and while the normal output is that of the raw diff it's very easy to wrap the results and add other checks on top of it. The shell's not written to be very defensive (unusual for me) but the code is short enough that it's worth the compromise.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/06 18:26 | /tools/commandline | Permanent link to this entry | This entry and same date
Mon, 05 Jan 2009
GUI config apps and a thousand cuts
Today has been one of those death by a thousand cut days. We did a
migration first thing in the morning (I'm not supposed to be awake at 6am
unless it's from a really late night) and while all the big bits were
planned and moved successfully the work list was missing enough little
pieces to make the rest of the day very annoying.
What made the work a lot harder was that the changes had to be made through a web front end that abstracted about 20 seconds of vim in to four minutes of clicking buttons that were never in the same place twice. It's been a while since I've had to bulk make production changes using this kind of interface so I was freshly amazed at how awful it was.
First of all was the time it took. The average change was about 8 mouse clicks, most of them on different pages, across a slow application that was working with a very large (for it) dataset. Second was the lack of a safety net. I had to do full copy and pastes to somewhere safe for each thing I wanted to change before changing it. It may not sound like much but if you come from the land of version control and diffing changes then it just feels so risky. And if you don't then I suggest you start learning one. Instead I had to rely on some hastily written post check scripts that confirmed the changes were correct when publicly viewed. We'd normally write these as a double check but without version control they become the single safeguard. Which were only effective after the change was made, which is better than nothing I suppose...
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/05 20:17 | /geekstuff | Permanent link to this entry | This entry and same date
Sun, 04 Jan 2009
Simple Stemming with Perl
Stemming is the process for reducing inflected (or sometimes derived)
words to their stem, base or root form.
-- Wikipedia article on
Stemming
Ever used a website that allowed you to tag content? Ever ended up accidently using slightly different tags? Something like graphs and graphing or blog and blogs? (I hope so, otherwise it's just me...) To spot some of the more obvious overlaps you can stem each of the words and look for a common base. Where one's found there is the possibility of mistaken duplication. For example if you passed hunts, hunted and hunting through a stemmer each would return 'hunt'. If you want to try for yourself there are online stemmers available.
As a more concrete example let's look at the wonderful service del.icio.us. You upload your own bookmarks, tag them with a number of keywords and can then group, sort and search them by your own defined terms. Except I have a habit of tagging articles about similar topics with nearly, but not quite the same tag.
The perl code below shows how easy it is (using Lingua::Stem from CPAN) to run your own data through a stemmer and look for overlaps. There are implementations in most languages (PyStemmer is also very nice) and the wikipedia article is actually a very easy to follow introduction.
#!/usr/bin/perl -w
use strict;
use warnings;
use Lingua::Stem;
use Net::Delicious;
my $del = Net::Delicious->new(
{
user => "username",
pswd => "password"
}
);
my $stemmer = Lingua::Stem->new( -locale => 'EN-UK' );
my %stems;
for my $tag ( $del->tags() ) {
my $stemmed = $stemmer->stem( $tag->tag );
push( @{ $stems{$stemmed->[0]} }, $tag->tag );
}
for my $stemmed (sort keys %stems ) {
# we only care about base words with more than one tag associated
next unless ( scalar @{ $stems{$stemmed} } > 1);
print "Possible duplicates -\n";
print " -- ";
print join(" : ", @{ $stems{$stemmed} }), "\n";
}
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/04 19:32 | /perl | Permanent link to this entry | This entry and same date
Sat, 03 Jan 2009
The Art of Capacity Planning - Short Review
The only books on capacity planning I've ever skimmed my way through have
been dense, dull tomes of long mathematical formulas, advice that's hard to
use in any practical way and page counts in the treble digits. Thankfully
John Allspaw has bucked this trend with The Art of Capacity Planning
and instead written a slender, thought provoking, book.
The main focus of the book is that measurement is good, blind guessing is bad and that capacity planning, like security, is an ongoing process. While a lot of the material is common sense - which is never that common in IT - it's a perfect introduction to capacity planning (and the principles of data collection and graphing) for novice to intermediate system administrators and a handy refresher for the experts in the crowd. I found it oddly reassuring that someone else has a lot of the same thoughts as I do when it comes to these topics.
The Art of Capacity Planning is an easy, engaging read that gets you thinking along the right lines without becoming dull or long winded. Well worth the couple of hours it'll take to read - 8/10
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/03 17:47 | /books | Permanent link to this entry | This entry and same date
Fri, 02 Jan 2009
New year, new laptop - Samsung NC10
Near the middle of December I lost a very dear, and constant, companion -
my Sony Vaio 'some model number or other'. After nearly five years the
laptop stopped charging and it wasn't worth paying for the repairs. I put
off getting a replacement for as long as I could but while I had the work
laptop as a standby I needed a machine I could treat as my own. Something
outside the company security policy. Something I could install lots of
applications and languages I'll only ever look at once on. So I bit the
bullet and bought myself a Samsung
NC10.
It's not exactly been a long time since I bought it so I've hardly stressed the machine too but first impressions are very favourable. Battery life on wireless is a good four-six hours (depending on what else I'm doing). The keyboard is much nicer than the Asus ePC I used for about ten minutes before cramping my hands up and the screens actually very usable. It'll never replace a dual monitor setup but it's fine for writing little scripts, web browsing and reading my email.
I've got a 1GB memory upgrade on order (it can only take 2GB) and then I'll see if I can make VMWare play nice without killing the battery.
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/02 21:22 | /geekstuff | Permanent link to this entry | This entry and same date
Thu, 01 Jan 2009
Erlang in Practice - PragProg Screencasts
I recently watched the first in the series of the Pragmatic Programmers Erlang in Practice Screencasts (by Kevin Smith - no, not that Kevin Smith). As I've not seen them discussed that much else where I thought I'd jot down my thoughts.
First up a disclaimer/warning - I'm not an Erlang person and despite the title of 'Episode 1' this series of screencasts is not aimed at people with no experience in the language. If you want to learn Erlang then I'd suggest you read Programming Erlang instead. Once you've been through the book then you should consider coming back to this series.
Now, to look at the screencasts from a different angle - production quality and value for money. Despite not knowing enough Erlang to understand all the code presented, I found the quality of the screencast to be perfect for watching on a laptop. The video was clear, the presenters voice didn't make me want to kill him (although this is a highly personal thing) and at five dollars the price was right for an hours worth of content.
So would I buy another one? Yes, but not this series. Until I get a chance to work my way through the Erlang book this series is off limits to me, The Ruby Object Model and Metaprogramming on the other other hand is mighty tempting for under five UK pounds...
Like this post? - Digg Me! | Add to del.icio.us! | reddit this!
Posted: 2009/01/01 20:05 | /programming | Permanent link to this entry | This entry and same date
Copyright © 2000-2010 Dean Wilson
