Mailing List CGatePro@mail.stalker.com Message #100636
From: Nicolas Hatier <nicolas.hatier@niversoft.com>
Subject: Re: Users are losing the INBOX
Date: Wed, 08 Sep 2010 18:30:50 -0400
To: CommuniGate Pro Discussions <CGatePro@mail.stalker.com>
Hello.

There is two different free scripts available on http://www.niversoft.com/scripts meant to convert mbox into mdir.

The first one - http://www.niversoft.com/products/cgscripts/#convert_mailboxes - is meant to convert all the mailboxes of one account on a live server. It's quite slow, and a bit risky as it will likely fail if someone access the account while it's working. Better do a backup of the account first and follow the instructions on screen carefully.

Basically, this script changes the account default mailbox type to the new format, then connects trough IMAP, and for each mailbox it creates a new one of the new type, moves all the messages, then deletes the old one.

The second one - http://www.niversoft.com/products/cgscripts/#fast_mailbox_conversion - is meant to convert a single mailbox independently of the server. It's recommended to run in when the server is stopped, but it's possible to run it on a live server too without harm, except for the INBOX.

This second script reads the mbox file directly and create mdir files on the fly. It doesn't delete the mbox file afterwards.

We have another series of scripts here to perform, with supervision, a controlled conversion of all the relevant mailboxes of all accounts on a server. We successfully converted several servers from mbox to mdir, and back, on a live system. We only needed to quickly shut down and restart the CGP server a few times during the session to ensure critical data was uncached. If this kind of service may interest you, please contact me offlist.

Regards
Nicolas Hatier


On 2010-09-08 18:16, Wiley Sanders wrote:
Hello all,

This is a problem that we have been having for a long time with both Communigate 4.X and the current 5.3 version. About twice per month, one of my 10,000 users loses all messages in their INBOX. All messages are alleged to disappear.

The problem is made more difficult because the users are usually my most clueless, and they usually cannot describe when the problem occurred or what actions they were taking when it happened. In many cases it happens overnight. At first I was convinced it was just clueless users, but I think at least some of them are victims of some kind of bug or another and not just accidentally selecting messages for deletion. In only a very few cases do I actually find "lost" email in their trash.

The one thing they all have in common is that they are all legacy users with MBOX-format mailboxes (all new users have been MDIR for several years.) They are almost certainly not using multiple clients on the same INBOX. The INBOX.mbox file is truncated to zero length, and new mail begins showing up right away in it. In come cases the INBOX.bdx file disappears. Most victims seem to be using the Webmail interface, in fact I have not been able to confirm that any are using Thunderbird or an IMAP client. Of course it is a PITA to restore an MBOX file since it has to be restored in its entirety and usually cannot just be prepended to the existing INBOX.mbox file.

Both the old and new versions of CGPro were running on Linux, the current on a brand-new up to date RHEL 5.X server, the older on SuSe Enterprise 9.

My question is: Is there a setting in CGPro, the Linux OS, or EXT3 I can make to do more aggressive file caching, and perhaps more atomic file locking?

Alternatively, is there a tool I can use to migrate a user from MBOX to MDIR format? This problem doesn't seem to happen with MDIR format inboxes.

Thanks,
-W Sanders
 St Marys College of California
 http://wsanders.net



--

Nicolas Hatier, ing. <nicolas.hatier@niversoft.com>
Niversoft idées logicielles - http://www.niversoft.com

Subscribe (FEED) Subscribe (DIGEST) Subscribe (INDEX) Unsubscribe Mail to Listmaster