APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed

Quick and simple web log grepper in Perl

© December 2010 Anthony Lawrence

I often want quick answers to very simple questions. How many pageviews and unique visitors were there for a certain page this month or yesterday?

Sure, Google Analytics can tell me that, though Analytics won't count visitors with Javascript disabled and sometimes misses even then. More importantly, if I don't already have it open, it takes several steps to get what I want.

Grepping raw log files always works, and is fast, but a Perl program is effective and gives me what I usually want very quickly.

You might need o have to adjust how the log is read - mine get compressed daily so I have to zcat them to get what I want.

Usage is simple enough: countuniq arg1 arg2 ...

Example "countuniq sample.html 16/Dec"

This ignores any GET of jpg's etc. Of course you could modify this to add all kinds of switches and features, but for me, it's a daily tool that is quick and simple.

chdir("/home/cluster1/data/g/q/1863165/logs") or die "No logs!";
@stuff=`zcat tr*.gz | cat - trans*[0-9]`;

READING: foreach (@stuff) {
 next if not /html *HTTP/;
 foreach (@ARGV) {
   next READING if ( $line !~ /$arg/i);
 s/ .*//;

$args=join " ", @ARGV;
print  "$args:  Unique IP: " . scalar (keys %uniques);
print  " Pageviews:  $pviews \n";

Got something to add? Send me email.

(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> Simple Web Log Grepper in Perl

Inexpensive and informative Apple related e-books:

iOS 10: A Take Control Crash Course

Photos for Mac: A Take Control Crash Course

Photos: A Take Control Crash Course

Take Control of Parallels Desktop 12

Take Control of Apple Mail, Third Edition

More Articles by © Anthony Lawrence

Printer Friendly Version

Have you tried Searching this site?

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us

Printer Friendly Version

Getting information off the Internet is like taking a drink from a fire hydrant. (Mitchell Kapor)

Linux posts

Troubleshooting posts

This post tagged:





Unix/Linux Consultants

Skills Tests

Unix/Linux Book Reviews

My Unix/Linux Troubleshooting Book

This site runs on Linode