"Home Directory Full" message "fix doesn't work for me

Question

Often (no, not always, it's not consistent), I get the dreaded "Home Directory full" message when I launch mail. Mail then quits when I click OK. Normally, if I launch it again right away, it works.
I tried to use the standard fix (e.g., removing the "Envelope Index" file to the desktop and launching Mail). This did cause a dialog to pup up informing me that I needed to re-index my mail file. I did that, and the process stopped with the progress indicator about two thirds through. I quit Mail and replaced the Mail folder in Library with the backup I had made before getting started.
Next, I tried the Hoover method (running the vacuum command from the Terminal. My machine grinded away for some time. I don't know how far it got (there is no progress indicator nor any other feedback of any kind), but since nothing happened for a good half hour (with no more disk activity for a long time), I quit that process, too. Get info revealed that the "Envelope Index" file had not been modified at all by this. Again, just to be on the safe side, I replaced the Mail file again with the backup copy.
Because the re-indexing in my first attempt had stopped while processing my sent items folder, I suspected that this one was the culprit. I therefore attempted to rebuild it (via the menu command in Mail). Much to my surprise, the process actually finished without an error message. However, my sent items folder now contained some 5,200 emails as opposed to the 7,00 or so it should have.
Then I saved my Mail preferences file to the desktop, made a backup copy of my Mail backup file, and launched Mail. This made the application believe that it was being launched for the first time. I used the Import command and imported only the contents of the Sent Items mailbox, specifying "Other" as the type. This imported 5,225 items; at that point, the progress indicator was showed a fully completed process, even though the "messages" folder contained 7,900 items. Checking things, I found that there was a discrepancy of 9 items (i.e., the last item that showed as having been imported was not the 5,225th item in the "messages" folder, but rather the 5,216th). I thus moved the first 5,216 items from the messages folder into a different folder and tried to import the rest. Much to my surprise, Mail imported the same 5,216 items all over again. This is when I noticed that it does not import from the "messages" folder but rather from the file called "mbox".
And this is where things stand. I still get the "Home Directory Full" message about once out of two launches of Mail (but never once Mail is up and running). I desperately need to get this fixed as I fear I'm on the brink of a disaster with my mail file.
One more clue: I cannot be 100% sure, but it seems to me that the problem began when I added a second dot mac email account to my Mail setup (we have a family pack; I always managed the main account and have more recently added one of the sub-accounts to my configuration). I also noticed that sometimes, this second dot Mac account shows up in my list of mailboxes on the left side of the screen, and sometimes it doesn't. Both these accounts are configured to be checked automatically. In addition to these IMAP accounts, I have a half dozen POP3 accounts and some 40 or 50 local mailboxes configured.
Does anyone know of a utility I can use to rebuild my entire email file instead of just the index? Or does anyone have any suggestions as to how I can manually fix things? I did do the usual stuff already (using disk utility to check the disk, fixing permissions, and so on) without detecting any problems.
Thanks in advance for any assistance you can offer.
Daniel

Answer 1

You’re welcome.
When I removed the preference file (com.apple.mail.plist), I also removed
the folder "Mail" so as to start from scratch completely.
Well, that’s not what you said in your first post...
I only use "Other" because I read in one of Apple's notes that one should
try "Other" in the event that the OS X Mail import failed (which it did).
This is wrong in the case of Mail 2.x. That has to be an old article. Could you please be more specific as to where exactly did you find it?
there is a way to actually export the contents of a mailbox as individual
items that can then be reimported.
Selected messages can be exported in standard mbox format by doing File > Save As in Mail and choosing Raw Message Source from the Format popup menu. Standard mbox files can be imported into most mail clients. In particular, they can be imported back into Mail doing File > Import Mailboxes and choosing Other as the data format. Using this format, however, would cause some status flags to be lost (e.g. whether a message has been replied to).
Locally stored mailboxes can be copied to the Desktop or anywhere else by just dragging them there from Mail itself (for server-stored IMAP mailboxes, copy the messages to locally stored “On My Mac” mailboxes in Mail first). These, however, are .mbox folders (not standard mbox files), where messages are stored in individual .emlx files that other mail clients don’t know or care about. They can be imported back into Mail doing File > Import Mailboxes and choosing Mail for Mac OS X as the data format.
It also explains why the import only brought in 5,200 items instead of the 7,900.
The question is: where are the others?
In the Messages folder.
as I mentioned above, I used disk utility to check the disk and fix permissions, so I did
not just do the permissions stuff. In my experience, if checking the disk gives you an
OK result, then there is no point in booting from another disk to perform a repair disk.
Right. I missed that part of your first post. Sorry.
if the mbox files are no longer needed, can they be removed?
Yes. The only items within a mailbox (i.e. within an .mbox or an .imapmbox folder) used by Mail 2.x are Info.plist and the Messages folder. Any other files you may see there are almost certainly Mail 1.x files that Mail 2.x leaves there after the conversion. This is poorly explained in Mac OS X 10.4 Mail: Some mailbox files used by Mac OS X 10.3 are not deleted after importing. The article only talks about mailboxes in ~/Library/Mail/Mailboxes/, but the same can also be said about mailboxes in “POP-”, “IMAP-”, or “Mac-” account folders.
As long as you keep a backup copy of the old files for a while (at least, until you’re sure everything was converted properly), they can safely be removed from the Mail folder. OnyX has a Cleaning > Misc > Temporary and obsolete items option for getting rid of all those files.
Note that if you see strangely-named Messages-T0x... folders, these aren’t old files from previous versions of Mail, but rather an indication that there is something amiss. More specifically, these are temporary folders created by Mail 2.x during an import or a reindexing process (e.g. rebuilding a mailbox) that Mail should have deleted when done. Their presence is a clear indication that something didn’t work as expected.
<hr>
Here’s the procedure I usually suggest to set up Mail again from scratch:
1. If you have a .Mac account and .Mac synchronization of Mail data is enabled either in Mail > Preferences > General or in System Preferences > .Mac, disable it before proceeding.
2. Quit Mail if it’s running.
3. In the Finder, go to ~/Library/. Move (not copy) the entire Mail folder out of there, to the Desktop.
4. In the Finder, go to ~/Library/Preferences/. Locate com.apple.mail.plist and move it to the Desktop.
5. Open Mail and set it up again from scratch. If given the option to import existing mailboxes or something like that, don’t. Just enter the account information and check that everything works fine.
6. You’ll have to re-configure all your settings in Mail > Preferences. For spam-related security reasons, the first thing you should do is go to Preferences > Viewing and disable Display remote images in HTML messages if it’s enabled.
7. If Mail works fine now and you had any messages stored in local mailboxes, do File > Import Mailboxes, choose Mail for Mac OS X as the data format, and follow the instructions to import your mail from the old Mail folder that’s now on the Desktop. Import the Mailboxes folder first, then each of the POP-username@mailserver account folders, if any.
8. If .Mac synchronization of Mail data was enabled at the beginning, enable it again, go to System Preferences > .Mac > Advanced, click Reset Sync Data, and choose the appropriate options to reset the Mail data stored on the .Mac server with the data locally stored on the computer, i.e. sync data must flow from the computer to the .Mac server.
As a result of doing the above, some messages may be duplicated. Andreas Amann’s Mail Scripts has a Remove Duplicates script that you may find useful.
If all is well and you don’t miss anything, the files on the Desktop can be deleted, although you may want to keep them for a while, just in case.
Note: For those not familiarized with the ~/ notation, it refers to the user’s home folder. You can easily locate any of the folders referred to in this post by copying the folder path here, doing Go > Go to Folder in the Finder, and pasting the folder path there.