Your personal GMail-like mail system: fetching with getmail

Now your e-mail storage is converted and you are ready to go. Hold on, it is time to do some preparation work. While not strictly necessary, I suggest you you figure up and write down the following information:

• Which user id will own the mail storage files. It will also run the mail retriever and (in future) the delivery agent. I suggest creating a dedicated UID for it.
• Which top directories would contain your e-mails.

I decided to create the user mailman, and create the storage in its home directories under the root user:

# useradd -m mailman
# mkdir -p /home/mailman/storage/business
# mkdir -p /home/mailman/storage/personal
# cp -r <old business email storage path> /home/mailman/storage/business/
# cp -r <old personal email storage path> /home/mailman/storage/personal/
# chown -R mailman /home/mailman/storage/

As you see, the mail is stored in two different directories inside the storage directory, and all the files are owned by the mailman user.

Once you set up the storage and the user, you’re ready to setup the mail fetching with getmail.

Installing getmail

Getmail is a Python script which retrieves e-mails from your mail providers and stores them into the Maildir folders according to configuration. You can use procmail instead, but I found getmail easier to configure and its ability to deliver to Maildir mail boxes natively is very useful in the early stages of building the e-mail system.

Installing getmail is typically as easy as installing the relevant package from your Linux distribution. However if your distro doesn’t provide one, you can always go to the getmail website and download it. Unpack it, and build and install it as described in README under the regular user:

> cd getmail-4*
> python setup.py build
> sudo python setup.py install

Configuring getmail

Getmail stores its configuration together with the internal cache files in a directory. Which means for each account you want to retrieve the mail from you will create a directory with the getmail configuration file getmailrc in it. So for my three mail providers I create three directories:

# su - mailman
> mkdir -p getmail/personal
> mkdir -p getmail/business
> mkdir -p getmail/gmail

and in each of those directories I created the getmailrc file. Since all my providers support POP3 retriever over SSL, the content of the getmailrc file was very similar to each of them. Note that if your provider requires you to use IMAP, you need to use a different retriever so check the getmail configuration.

Here’s the getmailrc configuration I use to retrieve email from my gmail account, which is stored in the /home/mailman/getmail/gmail/getmailrc file:

[retriever]
type = SimplePOP3SSLRetriever
server = pop.gmail.com
username = <gmail username>
password = <gmail password>

[destination]
type = Maildir
path = /home/mailman/storage/personal/

[options]
read_all = 0
delete = 0
delete_after = 3

As you see, for the sites which support POP3 over SSL all you need is to change the server, username and password in the [retriever] section. If the e-mail from this specific site should be delivered into a different storage (i.e. business), you also need to change the path in [destination] section but that’s all.

If your site supports POP3 but does not support SSL, use the type=SimplePOP3Retriever instead of using SimplePOP3SSLRetriever. If the POP3 server listens on a non-standard port number like 11000, you can also add a port=11000 parameter into this section. See the getmail documentation.

The options I use is to keep the mail on the remote server for three days, then delete it. This ensures the remote mailbox is not likely to overflow with mail, and provides me with some safety net knowing I can still access my recent email directly.

Testing your configuration

To test your configuration run getmail with the –getmaildir argument followed by the path to your getmail directory under the mailman user which can write into the maildirs:

> /usr/bin/getmail  --getmaildir /home/mailman/getmail/gmail

and you will see the output similar to the one below:

Copyright (C) 1998-2009 Charles Cazabon. Licensed under the GNU GPL version 2.
SimplePOP3SSLRetriever:gmail_username@pop.gmail.com:995:
 0 messages (0 bytes) retrieved, 0 skipped

If you receive any errors, correct them. For gmail, for example, you may need to enable POP3 access in your Account Settings if you never done it before. If you get write errors, make sure that the user you’re running it under has the permissions to write both into the maildir storage and into the current directory.

Once the configuration was successfully tested, use it as a template. Copy the getmailrc file into different directories, edit the relevant fields and test them.

Running it regularly

Since you probably don’t want to run getmail manually, I suggest using crontab for this task. Type crontab -e under the mailman user and create the following entry for each account you want to retrieve the mail from:

*/3 * * * *  /usr/bin/getmail -q /usr/bin/getmail  --getmaildir /home/mailman/getmail/gmail >/dev/null 2>&1

this line starts getmail every three minutes, and keeps it completely quiet. If you decide to remove the /dev/null redirections and your Internet connection goes down for a few hours, you’re going to receive a bunch of e-mails every three minutes reporting those connection errors, so beware.

Now you have the mail storage with current mail, and it is time to configure the IMAP server to access it.