APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed

Quick and simple web log grepper in Perl

I often want quick answers to very simple questions. How many pageviews and unique visitors were there for a certain page this month or yesterday?

Sure, Google Analytics can tell me that, though Analytics won't count visitors with Javascript disabled and sometimes misses even then. More importantly, if I don't already have it open, it takes several steps to get what I want.

Grepping raw log files always works, and is fast, but a Perl program is effective and gives me what I usually want very quickly.

You might need o have to adjust how the log is read - mine get compressed daily so I have to zcat them to get what I want.

Usage is simple enough: countuniq arg1 arg2 ...

Example "countuniq sample.html 16/Dec"

This ignores any GET of jpg's etc. Of course you could modify this to add all kinds of switches and features, but for me, it's a daily tool that is quick and simple.

#!/usr/bin/perl
chdir("/home/cluster1/data/g/q/1863165/logs") or die "No logs!";
@stuff=`zcat tr*.gz | cat - trans*[0-9]`;

READING: foreach (@stuff) {
 next if not /html *HTTP/;
 $line=$_;
 foreach (@ARGV) {
   $arg=$_;
   next READING if ( $line !~ /$arg/i);
}
 #print;
 $pviews++;
 s/ .*//;
 $uniques{$_}++;
}

$args=join " ", @ARGV;
print  "$args:  Unique IP: " . scalar (keys %uniques);
print  " Pageviews:  $pviews \n";
 


Got something to add? Send me email.





(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> -> Simple Web Log Grepper in Perl




Increase ad revenue 50-250% with Ezoic


More Articles by

Find me on Google+

© Anthony Lawrence



Kerio Samepage


Have you tried Searching this site?

Unix/Linux/Mac OS X support by phone, email or on-site: Support Rates

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us





Technology is anything that wasn’t around when you were born. (Alan Kay)

If it looks like a duck, and quacks like a duck, we have at least to consider the possibility that we have a small aquatic bird of the family Anatidae on our hands. (Douglas Adams)







This post tagged: