Script Gmail backup with getmail on linux

Backing up Gmail has been on my list of things to do ever since Google accidentally deleted a whole bunch of accounts. My inbox containts a lot of important information and if it were to be lost, it would be a major pain. I finally got around to backing it all up.

My requirements were:

  • Scriptable – I wanted it to run weekly with all my other backups, so no GUI or thick email client to backup to PST or something.
  • Accessible offline – If I wanted to go back to a set of emails that were lost, I could easily do it either with text utilities or a GUI application.
  • Download over IMAPS to get around Gmail’s 100 messages at a time POP3 download limitation.
  • Download only new messages.

I found getmail. I could cron it and it can store the mail in both mbox format for use with mail clients as well as individual message files.

Preparation – install getmail and prepare backup location

Installation is easy on an Ubuntu system:

sudo apt-get install getmail4

I’m sure there are equivalent package manager packages for other operating systems if Ubuntu isn’t your choice.

Now let’s create the backup directory structure. Create a subdirectory for the individual mail files called maildir. The maildir format requires three directories for processing mail (tmp, cur, new), so create those in the maildir. Lastly, touch the mbox file.

mkdir /mnt/mir/Backup/GmailBackup
cd /mnt/mir/Backup/GmailBackup
mkdir maildir
mkdir maildir/tmp
mkdir maildir/new
mkdir maildir/cur
touch backup.mbox

Build the getmail config file

I’m going to store the config in my Gmail backup directory as that will be backed up to my offsite. getmail wants it in your ~/.getmail directory though, so let’s create a link.

mkdir ~/.getmail
cd /mnt/mir/Backup/GmailBackup
touch backupgmail.getmailrc
ln -s /mnt/mir/Backup/GmailBackup/backupgmail.getmailrc /home/dan/.getmail/backupgmail.getmailrc

The contents of this configuration file are:

# define the settings used to connect to your mailbox
[retriever]
type = SimpleIMAPSSLRetriever
server = imap.gmail.com
username = your_email_here@gmail.com
password = $up3rSTr0ngP@SSw0rd
mailboxes = ("[Gmail]/All Mail",)  # this is the special Gmail label that signifies all messages when accessed through an IMAP client. If you only want to backup some labels, include them here.
port = 993

# main testination declaration.
[destination]
# specify that we want to back up to more than one place
type = MultiDestination
# and select the destinations
destinations = ('[mboxrd-destination]', '[maildir-destination]')

# mbox destination settings
[mboxrd-destination]
type = Mboxrd
path = /mnt/mir/Backup/GmailBackup/backup.mbox

# maildir destination settings
[maildir-destination]
type = Maildir
path = /mnt/mir/Backup/GmailBackup/maildir/

[options]
# prevent verbose command line output
verbose = 0
# do not re read messages that have already been downloaded (only download new messages)
read_all = false
# prevent getmail from adding a received header to the message as it is saved
received = false
# prevent getmail from adding a delivered_to header to the message as it is saved
delivered_to = false

Run it

If you have a large mailbox, the first run is going to take quite some time. I want to monitor it, so let’s give it the verbose flag, -v, so we see the message count increase as it works. Also, you need to specify the profile to use to download using the -r flag

$ getmail -v -r backupgmail.getmailrc
getmail version 4.14.0
Copyright (C) 1998-2009 Charles Cazabon.  Licensed under the GNU GPL version 2.
SimpleIMAPSSLRetriever:your_email_here@gmail.com@imap.gmail.com:993:
msg     1/20158 (4312 bytes) delivered
msg     2/20158 (3219 bytes) delivered
msg     3/20158 (10093 bytes) delivered
...

That’s it. Now you’re backing up and when complete, you’ll have a full archive of your Gmail. In order to copy down any new messages, create a crontab entry to backup daily at 11pm.

crontab -e
# add this line and save
0 23 * * * getmail -r backupgmail.getmailrc

Accessing your backup

So you have the backup, and you need to find some file that was deleted. The easiest way to get at your mail is to use Thunderbird.

  1. In Thunderbird, go to Tools -> Account Settings -> Local Folders and copy the setting of “Local directory”.
  2. Close Thunderbird.
  3. Copy the mbox file to the location you copied. If there is a .mbox extension on the file, remove it. so backup.mbox should just be backup.
  4. Open Thunderbird. On the left side under Local Folders you will now see a folder called backup with your mail.

4 thoughts on “Script Gmail backup with getmail on linux”

    1. Hi! I’m not sure what you mean. The script does kind of make a mess of the individual messages. I don’t plan on ever having to need to go through them but they are there as a kind of last ditch effort if my gmail went away and I absolutely HAD to find an old message. However, I my first plan would be to use the MBOX format file and put that in a mail client such as thunderbird. The individual files really are a last ditch effort, but could also be good for grepping.

      Hope that answers you question.

  1. Hi,

    Thanks for the answer. In fact I was wondering if you had found a convenient way to use maildir backup with a client. I know there is some support for maildir in thunderbird, but so far, I haven’t managed to use it.

    1. I haven’t spent much time looking into how to use the messages from maildir, sorry. If you find a good solution I’d love to hear about it!

Leave a Reply to Dan Cancel reply

Your email address will not be published. Required fields are marked *