APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed

Too much traffic


Some material is very old and may be incorrect today

© February 2009 Anthony Lawrence

This website limps along serving up a little less than 200,000 pages a month. If you divide that up, it's a bit over 6,000 a day, which comes down to 4 or 5 pages per minute. That's hardly a lot to ask of any webserver.

Of course sometimes we get peaks. There's not much happening at 3:00 AM so obviously there's more happening at noon. Every now and then some article gets a little burst of popularity despite my best efforts to discourage such things. Those cause peaks too.

But lately we've been getting some plain old peaks. I mean that they don't come because a particular page has attracted some attention; they come just because a lot of people came at once. For example, we had one just a few minutes ago and it hit just about one page view per second. As each page also has to pull in other stuff, the load on the server was three times that and it started throwing errors:

"unable to include /cgi-bin/blah-blah.pl", "Premature end of script headers:", "Permission denied".. not good.

Well, you can sympathise. If that load kept up all day and all night, I'd be serving over 2 million pages per month and that might be a little too much to ask of a shared web server.

So what to do? Well, I could use mod_perl. That's annoying, but it's probably what I have to do. Unfortunately, my code is far from ready for mod_perl so that will take some time and effort.

I could create static pages. That would be even more annoying.

I could ask more of you to use RSS. Go ahead and do that. I don't think it will help that much.

I could switch to a more expensive server. I really don't want to do that either.

Any other suggestions?


If you found something useful today, please consider a small donation.



Got something to add? Send me email.





(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

->
-> Too much traffic

5 comments


Inexpensive and informative Apple related e-books:

Are Your Bits Flipped?

iOS 8: A Take Control Crash Course

Digital Sharing Crash Course

Take Control of IOS 11

Take Control of iCloud, Fifth Edition





More Articles by © Anthony Lawrence







Thu Feb 12 21:40:58 2009: 5397   NickBarron

gravatar
How odd, I was just thinking about this yesterday.

How popular this site had become over the last year or so, even with your best efforts to discourage it ;)



Thu Feb 12 21:48:55 2009: 5398   TonyLawrence

gravatar
Funny, isn't it: everybody else is begging for more RSS subscribers, asking people to vote up their posts on Digg, sucking up to anyone who might give them a Tweet.. and I want exactly the opposite: keep the unwashed masses away and I'll be happy..

Really, traffic is fine and I will have to deal with it. Mod_perl is the simplest way I guess.. I just have to clean up my sloppy programming :-)







Fri Feb 13 13:52:17 2009: 5401   MikeHostetler

gravatar
If you didn't have so many things to say . . . .

mod_perl would work, but if your traffic keeps going up, you may have to deal with a "caching" solution that periodically generates static pages. This has become the accepted WordPress solution:
(link)








Fri Feb 13 13:57:24 2009: 5402   TonyLawrence

gravatar
Yeah, after sleeping on it I started thinking that rewriting to make static pages would be easier than rewriting everything for mod_perl. I'll start working on that today.



Fri Feb 13 17:10:12 2009: 5408   TonyLawrence

gravatar
OK, just finished up the code. I'm not going to convert everything at one - I'll test a few pages first to make sure I didn't bollix it.. so I'll test with few high volume pages and then roll it out..





------------------------


Printer Friendly Version

Have you tried Searching this site?

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us


Printer Friendly Version





Generally, old media don't die. They just have to grow old gracefully. Guess what, we still have stone masons. They haven't been the primary purveyors of the written word for a while now of course, but they still have a role because you wouldn't want a TV screen on your headstone. (Douglas Adams)




Linux posts

Troubleshooting posts


This post tagged:

Blogging

Web/HTML



Unix/Linux Consultants

Skills Tests

Unix/Linux Book Reviews

My Unix/Linux Troubleshooting Book

This site runs on Linode