I use GMail as my personal email provider, and as much as I like the simple and snappy UI, the conversation views and the filtering and search possibilities, I’ve grown more and more worried about having all of my (important) mails stored on someone else’s servers. Now, out of all of the big IT companies in the world I’d say I trust Google a heck of a lot more than anyone else, but this doesn’t mean that I shouldn’t take into consideration the possibility of something going wrong on their end and as a result some (or Bob forbid, all) of my mails disappearing into bit heaven.
The good news is that Google provides a nice, standard POP3 interface for downloading emails from their service, and all of the software required for downloading messages via POP is already installed in Mac OS X Leopard by default. Below I’ll go through all of the steps it took me to set up periodical and automated GMail backups on my Macbook.
The following were my sources for most of the information presented here:
- LifeHacker: Geek to Live: Back up Gmail with fetchmail
- Logic Bound: Backup Gmail Data
- bronski.net: fetchmail & SSL
- Matthias Andree and Paul Elliott on [fetchmail-users]: What is a “local issuer certificate”?
Configure GMail for POP access
- Go to GMail’s “Forwarding and POP/IMAP” settings
- Set the “POP access” setting to “Enable POP for all mail (even mail that’s already been downloaded)”
- Make sure that the “When messages are accessed with POP” setting says “keep Gmail’s copy in the inbox”
Configure fetchmail to Download the Messages
- Create ~/.fetchmailrc with the following contents (replacing GMAIL_USERNAME, GMAIL_PASSWORD and LOCAL_USERNAME with their real values in your case):
poll pop.gmail.com with proto POP3 and options no dns user 'GMAIL_USERNAME@gmail.com' there with password 'GMAIL_PASSWORD' is 'LOCAL_USERNAME' here and wants mda "/usr/bin/procmail -d %T" options ssl keep sslcertck sslcertpath "/Users/LOCAL_USERNAME/.ssl/certs"
This configuration basically tells fetchmail to:
- Connect to the GMail POP server using the POP3 protocol,
- Map your GMail username to your localhost username,
- Deliver the downloaded mail into your local system mailbox (mail spool file),
- Connect using an encrypted SSL connection,
- Strictly check the SSL certificates of the server it connects to against local trusted certificates,
- Search for the local certificates in ~/.ssl/certs
From the output of this command, copy the part that looks like this:
(a big bunch of characters here)
And save it into a file called ~/.ssl/certs/gmailpop.pem.
Optional: Specify the Location for Downloaded Messages
We’ve configured fetchmail to send messages to procmail, which will handle the storing of them in a mail spool file. By default this will be in /var/mail/LOCAL_USERNAME, but if you want to store it somewhere else, you can specify the location in the ~/.procmailrc configuration file. Below is an example (replace LOCAL_USERNAME with your username):
I prefer to exclude my mail spool file from Time Machine backups (because it’s a large file that changes very often which makes it take up a lot of disk space from my backup volume) so I have it located in ~/.mailspool/ like in the above example. This is because in OS X /var is actually a symbolic link to /private/var and Time Machine has a bug where it’s impossible to exclude anything from under that path from backups (for example, if I choose to exclude /private/var/mail in the GUI, it’ll replace this selection with /var/mail automatically, and even though /var/mail is then excluded, it’ll still back up /private/var/mail).
Download Messages with fetchmail
You can now download the mail by running this command (the -v argument is for verbose output):
Unless you only have a few mails in your Gmail box, this command will initially have to be run several times in a row in order to get all of the messages since it only downloads a few hundred messages at a time. The mail spool file containing all of the downloaded messages will then be found in /var/mail/LOCAL_USERNAME (or in whichever location you’ve specified in ~/.procmailrc).
It’s a good idea to set up a launchd job for running this command between regular intervals so that you wouldn’t have to remember to manually do it yourself. Lingon is an ideal GUI app for this purpose. My GMail backup launchd job doesn’t call fetchmail directly, though: it calls this helper Python script that I’ve written. The script does the following:
- Runs fetchmail,
- Displays Growl messages about the result (success or failure, number of messages downloaded if successful, etc.),
- Optionally saves fetchmail’s output into a log file when done.
Feel free to use it (remember to set the values in the “settings” section before running it, though).
Read Downloaded Messages with Thunderbird
In Mozilla Thunderbird, go to Tools > Account Settings… > Local Folders to get to the Local directory path. Go into this directory and create a symbolic link there that points to the mail spool file:
ln -s /var/mail/LOCAL_USERNAME ./GMailBackups
After this you’ll be able to find your backed up GMail messages in a folder called “GMailBackups” under “Local Folders” in Thunderbird.