Tag Archives: Simple Tools

Back to Basics – Very Simple Log Monitoring with Perl

multitail_apache

There are many many tools out there which allow you to monitor and view your system or networking logs in several different ways.  Sometimes though, you may find yourself looking for a specific feature that none of these tools currently provide.  Whenever your goals are very specific, and you don’t want to use a big feature-full program to accomplish a simple task, you may want to consider writing your own tool.

Below is a simple Perl program I wrote which does just that. All the requirements of the program are within the script itself (using the __DATA__ handle at the bottom of the file). The only thing you may need to install on your system to get this to work is the File::Tail CPAN package.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
#!/usr/bin/perl -w
 
use strict;
use File::Tail;
 
my @patterns = <data>;
my $file = File::Tail-&gt;new ("/var/log/syslog");
while ( defined(my $line=$file-&gt;read) )
{
    my $match = &amp;filter($line);
    if ( $match eq "no" )
    {
        print $line;
    }
}
 
sub filter ()
{
    my $line = $_[0];
    my $match = "no";
    foreach my $test (@patterns)
    {
        chomp($test);
        if ( $line =~ m/$test/ )
        {
            $match = "yes";
        }
    }
    return $match;
}
 
__DATA__
PROTO=UDP SPT=67 DPT=68
ACCEPT IN=br0 OUT=vlan1 src=192.168.0.111.*PROTO=TCP.*DPT=80
ACCEPT IN=br0 OUT=vlan1 src=192.168.0.102.*PROTO=TCP.*DPT=80
ACCEPT IN=br0 OUT=vlan1 src=192.168.0.111.*PROTO=TCP.*DPT=443
ACCEPT IN=br0 OUT=vlan1 src=192.168.0.102.*PROTO=TCP.*DPT=443
ACCEPT IN=vlan1 OUT=br0.*DST=192.168.0.101.*PROTO=UDP.*DPT=1755
ACCEPT IN=vlan1 OUT=br0.*DST=192.168.0.101.*PROTO=TCP.*DPT=1755
ACCEPT IN=br0 OUT=vlan1 src=10.100.0.1.*PROTO=UDP.*DPT=53
JBLLNXWKS dhclient
 
__END__
</data>

This program will monitor the end of the file (like the Unix ‘tail’ command) and check for new log entries. When it detects new lines in the log, it will filter those lines with the patterns defined at the end of the script (under __DATA__) and display anything it detects in the logs except those filter lines.

You’ll probably notice that the filter lines are regular expressions, which makes this script more powerful than doing filtering by simple full-string comparison.

Aside from simply printing the output to STDOUT, you could use regular expressions to pop pieces of each line into an array or hash, in order to do calculations, such as how many entries had a source IP of X, or destination port of Y, etc.

Its definitely a good thing to keep in mind that whatever software you could possibly need is already out there on the internet, and possibly open source. However, its also good to keep in mind that YOU can create a tool yourself to accomplish your specific task; all it takes is a little self-confidence, effort, and patience.