Mail 2.0: Problems with deleting Envelope Index file and rebuilding mailboxes

Posted by Pierre Igot in: Mail
March 14th, 2007 • 10:34 am

There has been a fair amount of discussion on-line recently about a “vacuuming” trick that can be used to make Mac OS X’s Mail application faster, apparently, for some people.

This blog post by ‘bbum’ provides a good summary and a good technical explanation.

The trick involves the file called “Envelope Index” that Mail automatically creates at the root level of your library’s “Mail” folder. This file is an index of all your mail, and over time it can accumulate junk data that is no longer relevant or no longer used. The procedure described supposedly helps get rid of that junk, and make the “Envelope Index” file meaner and leaner, hence improving Mail’s overall performance.

My own experience with all this raises more issues than it solves, however.

Because of my involvement in the AppleSeed project for Mac OS X 10.4 (which introduced Mail 2.0), I have known for a long time that the “Envelope Index” file is an index file that can be trashed. (Mail 2.0 had a number of issues in early builds of 10.4 and the index file had to be rebuild to fix some of the issues.)

When you quit Mail 2.0 and trash the “Envelope Index” file and then relaunch Mail, Mail 2.0 behaves as if it has just been launched for the first time on the machine of a Mac OS X user that had been using Mail 1.x until now. Instead of giving you immediate access to the application’s main mail viewer window, it displays a splash screen asking you to confirm that you want to “import” all your messages.

The terminology used here is somewhat misleading, in that this “importing” procedure has nothing to do with Mail’s regular importing feature, which can import mail from other e-mail clients and typically places the resulting imported messages in a special mailbox in your mailbox drawer. Here, “importing” actually means “converting Mail 1.x e-mail messages to the e-mail message file format used by Mail 2.0”—which is obviously required if you want to use Mail 2.0 as your e-mail client.

The “importing” procedure doesn’t actually place the resulting imported e-mail messages in a special mailbox. It just converts all your Mail 1.x messages to the Mail 2.0 format and then displays the result in the Mail 2.0 interface. What it also does at the same time is build an index of all your message files—the “Envelope Index” file. This index file is obviously required to make browsing your mailboxes faster, because Mail 2.0 creates a separate “.emlx” file for each and every e-mail message, and browsing mailboxes would be much slower, even on a fast machine, if Mail had to reread each individual message file every time you access a mailbox containing that message.

Of course, if Mail does this misleadingly labelled “importing” procedure because you’ve just trashed the “Envelope Index” file, it doesn’t actually reconvert the messages from the Mail 1.x format. All it does is rebuild the “Envelope Index” file in your library’s “Mail” folder for Mail 2.0.

What the trick discussed recently on the web involves is using commands in Terminal to “clean up” the “Envelope Index” file without actually rebuilding it from scratch (which can take a while).

My own experience with the trick, however, is far from convincing. I tried it and my “Envelope Index” file was indeed somewhat smaller—although not that much smaller. However, when I relaunched Mail, I found that all my messages where unreadable. They were listed in their respective mailboxes, but when I double-clicked on them to open them, in the body of the e-mail instead of its actually contents I got a short message in italics telling me that Mail was unable to load the contents of the e-mail and asking me to take the account on-line.

Now, I should stress that I have a number of e-mail accounts in Mail, and that most of them are POP accounts, which means that this business about having to take the account on-line in order to be able to read the messages makes no sense whatsoever. Once I have retrieved my messages from the POP servers, they are on my machine here at home and whether the account is on-line or off-line makes no difference. I should be able to read my messages.

In any case, my accounts were already on-line, so there was nothing that I could do to force Mail to allow me to read my messages again.

I then quit Mail, trashed the “Envelope Index” file that had been produced by the “vacuuming” trick, and relaunched Mail. This forced Mail to go through the “importing” procedure mentioned above. I figured that, if the “vacuuming” trick had not worked properly, at least trashing the “Envelope Index” file should do the trick, in a more radical fashion.

Mail did indeed rebuild my “Envelope Index” file, and my messages were readable again, but then I soon noticed all kinds of weird stuff. In addition to the messages that were supposed to be there, my Inbox also contained a number of much older messages that I had deleted or filed away a long time ago. It also contained some duplicates of recent messages that were still supposed to be in my Inbox, but not in two copies. And all these mails were marked as unread.

In addition, while browsing other mailboxes in my mailbox drawer, I discovered that a number of e-mail messages that I filed away in various mailboxes a long time ago were flagged as unread, even though all the messages that I file away are always marked as read.

At that point, I figured that this might not have been a good idea after all, and I decided to quit Mail and restore my original (if slightly bloated) “Envelope Index” file, of which I had kept a copy on my desktop, just in case.

I did just that, and relaunched Mail, but unfortunately there were still problems with duplicate messages and messages flagged as unread, etc.

So I finally quit Mail again, trashed the “Envelope Index” file again, relaunched Mail and let it go through its “importing” procedure once more.

Again, the result was a mail environment with all my e-mail messages readable, but with some duplicates, some older messages in my Inbox that were supposed to be long gone, and some messages marked as unread in various mailboxes. I figured that I would just clean up the additional mess manually and forget about trying to understand what was going on exactly.

That’s when I also discovered another problem, however. When I found a mailbox that contained some messages that were wrongly flagged as unread, I would select the entire list of messages in the mailbox and then flag them all as read. Then, out of curiosity, I would try to use the “Rebuild” command in the “Mailbox” menu, as a final step to ensure that the mailbox was now fine and up-to-date.

Much to my surprise, using the “Rebuild” command would once again cause the same messages to be flagged as unread! I tried repeatedly, and the results were invariably the same. No matter how many times I marked all the messages as read, as soon as I applied the “Rebuild” command to the mailbox, some of the messages (always the same ones) would be flagged as unread again.

I did some research in Apple’s Discussions forums and found that I was not the only one. Reading these discussions, it does look like Mail’s “Rebuild” command is somewhat flawed, and that there is not much that can be done about it.

This is a problem that is separate from the “vacuuming” trick and the stuff with the “Envelope Index” file, but the two seem related just the same. After all, deleting the “Envelope Index” file, at least on my machine, also causes some messages to be flagged as unread again (along with other problems). I am not sure what the “Rebuild” command does, but presumably it does make changes to the “Envelope Index” file too (by rebuilding the index for that particular mailbox), which would explain the connection.

In any case, I guess that, at this point, I should just give up on the whole thing. I have now trashed my “Envelope Index” file and forced Mail to rebuild it from scratch, so if I was going to see any performance benefits, I would have noticed them by now. (I haven’t noticed any.) And clearly no matter how many times I try to use the “Rebuild” command, I am not going to be able to prevent Mail from flagging some read messages as unread again and again.

So it’s a case of leaving good enough alone. My “Mail” folder in my library weighs nearly 1 GB and contains hundreds of thousands of message files. I wish that Mail were faster, but I don’t think there is anything that I can really do to make it so. I don’t like the idea that some of my e-mail messages seem permanently “damaged” in a way that causes Mail to flag them as unread again and again, but apart from that, the messages appear to be intact and are included in Spotlight searches, so I am not going to worry about it too much.

But it certainly is not something that inspires a huge amount of confidence in Mail. I am going to continue to religiously back up my “Mail” folder, just to be on the safe side. At least with Mail messages are stored as individual files, so there is no real reason to worry about ominous “database corruption” or anything like that—which can be a very real problem with other e-mail clients that store all your e-mail in a single, gigantic database file.

I am not going to try that “vacuuming” trick again any time soon, however.

One Response to “Mail 2.0: Problems with deleting Envelope Index file and rebuilding mailboxes”

  1. ssp says:

    My impression was also that the ‘vacuuming’ trick gave results that were either placebo or not very long lasting. A shame, really, as Mail could do with better speed when switching mailboxes.

    I also saw the problem of deleted messages turning up again in unread in some upgrades. My assumption is that perhaps they come from sessions where Mail crashed at some stage before the message file was deleted but when it was removed from the catalogue only. When re-scanning the Mail folder it rediscovers them and adds them to the folder. (Anyway, that’s better than the other way round, with messages disappearing…)

Leave a Reply

Comments are closed.