Kerio Connect Automatic Index Maintenance

The latest Kerio Connect Mailserver release includes both on demand and automatic reindexing of users mail folders. I decided to abuse that and see what happens.

To make a proper mess for the indexing to clean up, I first created a temporary folder and then used "scp" to copy a good sized INBOX into it.


mkdir  ~/safehold
cd  /opt/kerio/mailserver/store/mail/
aplawrence.com/tony/INBOX
scp  -r  .  ~/safehold
 

Now it was time to mess things up:


rm   ~/safehold/?mgs/*
cd /opt/kerio/mailserver/store/mail/
aplawrence.com/tony/INBOX/?msgs/
for i in 0000605*.eml; do mv $i ~/safehold/?msgs/; done
mv ~/safehold ./TESTBOX
 

What all that did was create a new mailbox called TESTBOX that only had 16 messages in it, but had indexes that were remembering many thousand. It also rudely removed those same 16 messages from the original INBOX.

So what did Kerio see when I logged in?

Kerio mailbox index rebuild 1

It thought TESTBOX had thousands of messages in it, but it couldn't actually find any of them. It also didn't know that 16 messages had left the INBOX without permission.

However, just a few minutes later it had found the 16 messages, though it was still confused on the left about the count. As I watched it over several minutes, the count slowly decreased.

Kerio mailbox index rebuild 2

Kerio mailbox index rebuilding

I decided to see what would happen if I asked for a manual rebuild. You simply go to Users and, under More Actions, you'll find the option to Rebuild Mailbox. Oddly, that logs to the Config log (I would have thought Operations made more sense, but it didn't take very long.

Kerio mailbox index manual rebuild

Kerio mailbox index config log

Once complete (which took only seconds) , Webmail showed everything correctly. I wasn't done messing with it yet, though.

cd /opt/kerio/mailserver/store/mail/aplawrence.com/tony/INBOX/?msgs/
find . -mtime +15 -exec rm {} \;
 

That removed some 5,000 messages from right under Kerio's nose!

As before, it took a while for it to notice. The first hint was that if I tried to open one of the oldest messages, it came up blank.

Then whole pages disappeared. It still thought it had 39 pages of messages, but some were blank.

Kerio mailbox index rebuild in progress

After less than half an hour, all was well.

One final test - I removed 6 of the .eml files from TESTBOX/#msgs after logging out of webmail. I wanted to see if they background process would notice the missing files if I were not logged in.

My reason for that was that I wanted to see if it would be possible to rearrange a users mailbox during the night and have them see the result when they logged in the next day.

It was, although it took a while for it to happen.

# ls -l
total 584
drwx------ 2 root root   4096 2011-11-15 08:37 #assoc
-rw------- 1 root root 185994 2011-11-15 08:59 deleted.fld
-rw------- 1 root root   1723 2011-11-15 09:01 index.fld
drwx------ 2 root root 315392 2011-11-15 11:02 #msgs
-rw------- 1 root root   8192 2011-11-15 08:36 properties.fld
-rw------- 1 root root  59392 2011-11-15 09:01 search.fld
-rw------- 1 root root    158 2011-11-15 09:01 status.fld
 

Ninety minutes later I checked again:

drwx------ 2 root root   4096 2011-11-15 08:37 #assoc
-rw------- 1 root root 185994 2011-11-15 08:59 deleted.fld
-rw------- 1 root root   1084 2011-11-15 12:21 index.fld
drwx------ 2 root root 315392 2011-11-15 11:02 #msgs
-rw------- 1 root root   8192 2011-11-15 08:36 properties.fld
-rw------- 1 root root  44032 2011-11-15 12:21 search.fld
-rw------- 1 root root    158 2011-11-15 12:21 status.fld

Kerio mailbox index rebuild done overnight

It seems that the quickest way is a manual rebuild, that being active in the account will fix things up slowly, and that the background process will get around to fixing things eventually.

So, given this experiment, it seems possible that we could run a night time process that rearranged messages and have everything be fine in the morning. This could be useful for sorting out Inbox messages by month, for example,

I'll do a little sample code soon and we'll see how that works.



Got something to add? Send me email.





(OLDER) <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> -> Kerio Connect Automatic Index Maintenance


4 comments



Increase ad revenue 50-250% with Ezoic


More Articles by

Find me on Google+

© Anthony Lawrence







Thu Nov 17 10:52:13 2011: 10200   NickBarron

gravatar


Thanks Tony, I've been wanting to see how robust this is!







Thu Nov 17 11:25:13 2011: 10201   TonyLawrence

gravatar


It seems to be very good. Only time will tell, of course, but so far it looks like it is doing the job.



Tue Nov 22 15:34:45 2011: 10220   TomasSoukupKerio

gravatar


Hi Tony,

There is one important note to mention.
For clients that sync data to local cache (Outlook, Apple Mail, etc.), it can apply much later - or sometimes never. It depends when the client fully resyncs folder(s).



Tue Nov 22 20:53:21 2011: 10221   TonyLawrence

gravatar


Good point.

Shouldn't the background indexer be able to notify at least the KOC client that it had to fix indexes? Can that in turn tell Outlook to invalidate its cache?

I personally dislike Outlook and Apple Mail. Unfortunately, some folks insist upon using them.

------------------------
Kerio Connect Mailserver

Kerio Samepage

Kerio Control Firewall

Have you tried Searching this site?

Unix/Linux/Mac OS X support by phone, email or on-site: Support Rates

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us





Writing in C or C++ is like running a chain saw with all the safety guards removed. (Bob Gray)

Several people have told me that my inability to suffer fools gladly is one of my main weaknesses. ((Edsger W. Dijkstra)








This post tagged: