Additional Info





Computers have been taught to distrust each other and will reject attempted connections most of the time. Nowadays, most computers and firewalls are utterly rude about it: it would be like asking someone to dance and having them ignore you as though you were invisible and inaudible. (Tony Lawrence)

We are questioning more than the philosophy behind our dependence upon limited and limiting systems. We question the power structures that have grown up around such systems. (Frank Herbert)








This post tagged:



Share

Filtering and separating Kerio Control Debug logs


2012/10/30

Kerio debug logs can contain a lot of extraneous noise. Use a script like this to separate the wheat from the chaff.


Recently I have been working on a confusing troubleshooting problem involving Kerio Control VPN's. They had a working. albeit slow, VPN through a Cable ISP. We wanted to replace that with a faster VPN going through a fiber optic path, but that VPN was actually working more slowly than the supposedly slower Cable connection. Because working slowly is better than not working at all, it was often difficult to perform debugging tests.

The physical configuration was certainly partly at fault. Everything was carried through the LAN switches. That is, both WAN connections were plugged directly into the same switches that the LAN used and no VLANs separated them.

Packets dropped for some reason

When there is any sort of performance problem, we often turn on enhanced debugging. For situations like this, "Packets dropped for some reason" seems a reasonable place to start.

Adding packet drops to the Debug log

Unfortunately, that often gives us much more than we want. In this case, with multiple networks mixing their packets in, the noise was even worse than usual. I therefore turned to a Perl script to help me make more sense of what was happening.

The script presented here is a model of what you'd do to filter and separate a noisy log file. It simply tests for certain patterns and writes those patterns to separate output files. For example, NETBIOS traffic is generally of little interest, but it will be scattered throughout the log. Stripping those messages out to their own text file helps us zero in on other problems. The same is often true for any kind of broadcast message: we may want to look at them, but putting everything in separate files makes it easier.

Two important things to note: first, the patterns I used here are not necessarily what you'd use. You'd probably examine the original log file to determine just how you want to break things out.

Secondly, the order in which you test patterns can matter. For example, the netbios packets also include broadcast packets. Which file those end up in depends upon whether we first test for netbios packets or first test for general broadcast traffic.

Any log entries which are not matched are output to the screen. If your test patterns include everything you expect and are NOT interested in, the screen output will be the unexpected and therefore most interesting. Of course you can also add more patterns until nothing goes to the screen. If you do that, the line and byte counts of the files produced should end up equal to that of the original log - if not, you've made a programming error somewhere.

The handling of "Last message repeated" lines is simplistic and perhaps unneeded for all but broadcast traffic, but I put it in just in case.

#!/usr/bin/perl
use strict;
my $lmflag;
open(BCAST,">bcast.txt");
open(MCAST,">mcast.txt");
open(NETBIOS,">netbios.txt");
open(TRULE,">trule.txt");
open(IPS,">ips.txt");
open(ICMP,">icmp.txt");
open(TUNNELED,">tunneled.txt");
while (<> ) {
 if (/Last message repeated /) {
   print BCAST if $lmflag==1;
   print MCAST if $lmflag==2;
   print NETBIOS if $lmflag==3;
   print TRULE if $lmflag==4;
   print IPS if $lmflag==5;
   print ICMP if $lmflag==6;
   print TUNNELED if $lmflag==7;
   print if not $lmflag;
   next;
 }
 if (/-> [0-9]+.[0-9]+.[0-9]+.255:/) {
   print BCAST;
   $lmflag=1;
   next;
  }
 if (/-> 224.0/ ) {
   print MCAST;
   $lmflag=2;
   next;
  }
 if (/[0-9]+.[0-9]+.[0-9]+.[0-9]:13[7-9] ->/) {
   print NETBIOS;
   $lmflag=3;
   next;
  }
 if (/filtered and dropped by traffic rules/ ) {
   print TRULE;
   $lmflag=4;
   next;
  }
 if (/IPS Packet/ ) {
   print IPS;
   $lmflag=5;
   next;
  }
 if (/ICMP packet/ ) {
   print ICMP;
   $lmflag=6;
   next;
  }
 if (/tunneled/ ) {
   print TUNNELED;
   $lmflag=7;
   next;
  }
$lmflag=0;
print;
}

When adapted for your own needs, this type of script can help you cut through extraneous log noise to get at what you really want to see. At the same, it preserves everything else so that you can look at every type of message should you need to.



Got something to add? Send me email.





(OLDER) <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> -> Filtering and separating Kerio Control Debug logs




Increase ad revenue 50-250% with Ezoic


More Articles by

Find me on Google+

© Anthony Lawrence



Kerio Connect Mailserver

Kerio Samepage

Kerio Control Firewall

Have you tried Searching this site?

Unix/Linux/Mac OS X support by phone, email or on-site: Support Rates

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us