I often want quick answers to very simple questions. How many pageviews and unique visitors were there for a certain page this month or yesterday?
Sure, Google Analytics can tell me that, though Analytics won't count visitors with Javascript disabled and sometimes misses even then. More importantly, if I don't already have it open, it takes several steps to get what I want.
Grepping raw log files always works, and is fast, but a Perl program is effective and gives me what I usually want very quickly.
You might need o have to adjust how the log is read - mine get compressed daily so I have to zcat them to get what I want.
Usage is simple enough: countuniq arg1 arg2 ...
Example "countuniq sample.html 16/Dec"
This ignores any GET of jpg's etc. Of course you could modify this to add all kinds of switches and features, but for me, it's a daily tool that is quick and simple.
#!/usr/bin/perl chdir("/home/cluster1/data/g/q/1863165/logs") or die "No logs!"; @stuff=`zcat tr*.gz | cat - trans*[0-9]`; READING: foreach (@stuff) { next if not /html *HTTP/; $line=$_; foreach (@ARGV) { $arg=$_; next READING if ( $line !~ /$arg/i); } #print; $pviews++; s/ .*//; $uniques{$_}++; } $args=join " ", @ARGV; print "$args: Unique IP: " . scalar (keys %uniques); print " Pageviews: $pviews \n";
Got something to add? Send me email.
More Articles by Anthony Lawrence © 2010-12-20 Anthony Lawrence
Getting information off the Internet is like taking a drink from a fire hydrant. (Mitchell Kapor)
Printer Friendly Version
Quick and simple web log grepper in Perl Copyright © December 2010 Tony Lawrence
Have you tried Searching this site?
This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.
Contact us
Printer Friendly Version