Splitting Syslogs by Facility
Logs are a wonderful thing. If done correctly they point out the source of all errors, show you what’s running slow and contain useful information on how your system is running. At every place I’ve ever worked they’ve been busy, full of odd one offs and too often overlooked.
I’m going to be doing a fair bit of log processing next week so expect lots of little toolchain scripts like syslog-splitter.pl to be checked in to git and mentioned here.
syslog-splitter takes a logfile as an argument and breaks the logfiles
in to many smaller units, one file per facility (which contains all the
lines for that facility from the logfile), to make it easier to process. I
seem to invoke it followed by
wc -l out/* | sort -nr when on
new machines to work out where I need to invest some time. Over the next
week or so I’ll come back to the topic and show how I’m reducing the noise
to help me find the important lines.