Quick and simple web log grepper in Perl

I often want quick answers to very simple questions. How many pageviews and unique visitors were there for a certain page this month or yesterday?

Sure, Google Analytics can tell me that, though Analytics won't count visitors with Javascript disabled and sometimes misses even then. More importantly, if I don't already have it open, it takes several steps to get what I want.

Grepping raw log files always works, and is fast, but a Perl program is effective and gives me what I usually want very quickly.

You might need o have to adjust how the log is read - mine get compressed daily so I have to zcat them to get what I want.

Usage is simple enough: countuniq arg1 arg2 ...

Example "countuniq sample.html 16/Dec"

This ignores any GET of jpg's etc. Of course you could modify this to add all kinds of switches and features, but for me, it's a daily tool that is quick and simple.

chdir("/home/cluster1/data/g/q/1863165/logs") or die "No logs!";
@stuff=`zcat tr*.gz | cat - trans*[0-9]`;

READING: foreach (@stuff) {
 next if not /html *HTTP/;
 foreach (@ARGV) {
   next READING if ( $line !~ /$arg/i);
 s/ .*//;

$args=join " ", @ARGV;
print  "$args:  Unique IP: " . scalar (keys %uniques);
print  " Pageviews:  $pviews \n";

Got something to add? Send me email.

(OLDER) <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> -> Simple Web Log Grepper in Perl

Increase ad revenue 50-250% with Ezoic

More Articles by

Find me on Google+

© Anthony Lawrence

Kerio Samepage

Have you tried Searching this site?

Support Rates

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us