Your personal GMail-like mail system

My readers remember that I switched to Thunderbird back in December. This was quite difficult experience which included a lot of trial and errors, and although it didn’t result in any e-mail corruption or loss, this was definitely not kind of experience I’d like to go through in future. This made me wonder whether it is possible to avoid it altogether. I wanted a solution which would be:

• Independent of the e-mail client I am using. If tomorrow I decide to switch to a different e-mail client it should take no more than 15 minutes and should not result in any e-mail conversion or loss.
• Accessible and synchronized across multiple devices. I want to access all my email from my desktop e-mail client no matter which one of three OS I’m booted in. I want to access it from a Mutt running on my server, from my laptop, from my phone and from my tablet. And I want to see the consistent picture from all of them.
• Accessible on a single IP/port no matter how many accounts I want to support. This makes it possible to access my e-mail from various Internet cafes, airports, hotels, hospitals and so on which often provide very limited connectivity. For example, blocking the SMTP port is very typical in those environments, and some only provide access to HTTP and HTTPS ports.
• Flexible, easy to use and setup, working on Linux and backed up regularly on my schedule.

A word about GMail. Someone suggested switching to some cloud e-mail servirce, import old e-mail from all other accounts there and access it via IMAP. However this approach was rejected because of several reasons:

1. I want to keep the business and personal e-mails separately, i.e. on different accounts so some of my mobile devices would be configured to access only the business account but not the personal account. While this could be achieved with two GMail accounts, this would make the management more difficult.
2. There are business requirements related to how the e-mails should be kept or removed. For example, if I leave the company, all the e-mails related to that company are supposed to be deleted, but there may be a case when some may not be deleted – for example if there is a pending litigation. GMail is not flexible enough to provide this kind of granularity.
3. Lack of any kind of service guarantee for the free cloud e-mail users. I’m sure they have backups and their staff is taking care of the infrastructure, but if something happens tomorrow and they lose my mail, there is nothing I could possibly do.
4. In case of GMail the Google privacy record doesn’t look very impressive to me although I’m sure they changed for good.
5. As we recently seen with MegaUpload case, the whole service was shut down by the government because some of their customers used it to spread the content illegally. Which means any cloud storage available there is at least unreliable. What if other users of the same e-mail cloud service decide to use the service to discuss terrorist acts, sell drugs or for some other illegal activity? The government will shut down the whole service, as they did with MegaUpload. And you’ll lose access to all your e-mail, the same way the legitimate MegaUpload users lost the access to the content they owned. Sure, you can petition the FBI and the courts, and maybe after spending six months and hundred thousand dollars on lawyers you’ll get your old e-mail back, but I have stated it before and I’m restating it: in the current legal climate anyone relying on a multi-user cloud storage is a fool.

So the solution was to create the GMail like personal e-mail system for the limited number of users, which was not based on GMail, which would satisfy the following requirements:

• All e-mail content must be stored locally (i.e. on my own local server).  This makes import and backups easier, it makes the storage costs no concern, and it avoids liability by the third parties.
• E-mail must be stored in maildir format to make the system scalable, to have no concerns about possible file corruptions and to be able to irreversibly wipe an individual email if needed, without going through extremes.
• E-mail must be stored in two different storages, a business storage and the personal storage. This is necessary because the backup and retention policies are different.
• E-mail must be retrieved into the storage automatically from different accounts and stored in the appropriate storage. For example, all email from all ten personal accounts must go into the same storage.
• The e-mail should be accessible through IMAP, with two accounts, a business account and the personal account. All business e-mail should be collected automatically from different business accounts and stored in one local business account. So should personal email.
• Those accounts, however, are virtual. The storage uses only one system account. This makes the maintenance easier.
• The email accounts should have different access credentials and never got mixed.
• The mail system should support sending e-mail via SMTP, and should properly use the relevant SMTP server depending on which account the e-mail was sent from.
• The mail system should be exposed to public Internet and therefore require SSL/TLS encryption. SMTP also should require authentication.
• The mail system should support Sieve with an easy interface (preferably the one I can manage right from Firefox)
• There should be some cool Web interface which can access the e-mail for both accounts in case I want to check mail from some Internet cafe somewhere.
• All components must be free software and current, with the good security track.
• The system must run on a single Linux server, and shouldn’t take too much resources.

Going ahead I can say I succeed, and the solution I created satisfied all my needs. In an overview, the result system is composed of several free software components responsible for different areas:

• getmail is run as a crontab job and is used to retrieve e-mail from different various accounts via POP3 protocol, and store it in an associated account in the maildir structure.
• dovecot provides IMAP access as well as Sieve management capabilities. It supports virtual users, SSL authentication and modern IMAP extensions.
• postfix provides SMTP support as well as authentication and SSL. It works as SMTP forwarder and selects the proper destination SMTP server depending on the account.
• roundcube provides the Webmail functionality through a really cool web interface