People will often find you and your website through Internet searches. When they do, your log file will usually contain information on what they were searching for when they found you.
The following is a typical log entry showing a visit that came from a Google search. I bolded the "q=" section that shows the actual search string:
126.96.36.199 - - [05/Oct/2009:02:23:50 +0000] "GET /Forum/TonyLawrence13.html HTTP/1.1" 200 20191 "https://www.google.com.sg/search?hl=en &client=firefox-a&rls=org.mozilla%3Aen-US%3Aofficial&hs=agd &q=qmail+queues+stats&btnG=Search&meta=&aq=f&oq=" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:188.8.131.52) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729)"
Bing and many other search engines also use the "q=" format, so it's easy to mine this data from your logs.
This is obviously very useful information. Unfortunately, the search engine may not be sending your visitor to the page you'd really like them to hit. The page they got may indeed be related to their search, but when you as a human look at what search words they actually used, you would have sent them to a different page.
I look for that in my logs. When I spot something like that, I'll update the found page with a link and text that suggests the other page as possibly being something of interest also. Note that I don't want to just redirect to that page, because for other search keys the original page might be a perfect match.
Or I may find that all I need is to pull something from another page to update the page the search engines favor. The important thing is to help your visitors find what they need - if you aren't reading your logs, you don't know that they are or are not.
In the log entry shown above, the person was searching for "qmail queues stats". The page Google picked is appropriate in that case. However, I often find searches that aren't quite correct.
Analysing these logs can be a bit boring - often the search engines pick exactly what I would have picked. However, I find small problems often enough to make the effort worthwhile.
Got something to add? Send me email.
More Articles by Anthony Lawrence © 2009-11-07 Anthony Lawrence
Zawinski's Law: Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can. (Jamie Zawinski)