🎉 Email – it’s finally here!
Today we’re launching the modern, secure email system that we told you about back in March. To say that we’re happy to have this out is a huge understatement. I think of all the support tickets, tweets, and emails we’ve received “when will you have email?” has been the most asked question.
I know we’re about a month past our original time frame and a couple weeks past our revised estimates. If you want to know more about what took so long you can look at the very end of the post.
Here’s a quick reminder of the features of your new email system
🔐 Security:
- End to end TLS encryption. From the time the mail arrives at our MX servers until it’s delivered to your mailbox it is TLS encrypted.
- DKIM signing. When your email shows up in a Gmail inbox (or many others) it will now display a signed-by address of your domain. And the best part is the process is automatic (if you use our DNS servers).
✅ Deliverability:
- Decreased spam. We’ve put a lot of effort into the tools that allow us to identify and quarantine spam messages. And with your help we’ll train the spam filtering system even better.
- DKIM signing. With DKIM signing major email providers will know that you sent that email.
🗂 Mail filtering:
- Spam scanning with rspamd. Pre-trained with the latest data from the rspamd project and ready to continue learning as more spam data becomes available.
- Mail filtering with procmail. Bring your procmail filtering rules with you.
⚙️ Other features:
- Sendmail is enabled on all servers so your PHP applications (and many others) can send mail with no additional configuration.
- Easy migrations. Bring us your Maildir directory or use IMAP to copy your email over.
- Roundcube webmail. Your email available on the web.
My favorite parts of the email system are the backend tooling. We know from experience that managing outgoing spam attacks from bad actors and hacked sites can take hours. Our tooling allows us to do the same management in minutes.
That’s what you already knew about the system.
So what don’t you know?
🌐 You get regional email!
- Launching in the US and Germany today! Your email will be closer to you and much faster as a result. No more waiting for email to load.
- You can chose the region where each mailbox is located.
- Your email delivery will be more reliable with multiple SMTP servers and better management tools.
🤔 So what else do you get?
- No restrictions on the number of mailboxes.
- 1G of email storage — shared between all mailboxes.
- $5/month for 15G of additional shared mailbox storage.
🚀 How do you get started?
- Learn about mail users and email addresses in our control panel documentation.
- Learn about other aspects of the email system in our Email Topic Guide.
- You can use IMAP to sync your data over to our platform.
- You can have your old host provide your Maildir directory as an archive, upload it to your hosting account, and let us know in a support ticket and we’ll take it from there.
It’s been a long time in development (the entire time the company has been around now that I think about it) but it’s definitely worth the wait. The detours we took along the way ended up creating a better, more robust system.
I also can’t say enough good things about the team. We’ve had some frustrating last minute changes, late nights building and rebuilding things we thought were finished, and documenting and re-documenting things.
💤 What took so long?
tl;dr Development estimation is hard, complex systems have complex interactions.
When we wrote about the email system we had already finished a static, proof of concept build out of the email system on VMs. We were sure that the system would be able to accept and deliver mail, do DKIM signing, and work with the legacy mail systems most people are using.
The old saying “the map is not the territory” comes to mind here. When it came to putting together the production system we realized that without a working DNS system it would be extremely difficult to do any mail routing. So we had to stop and build out the DNS system.
Next there was huge demand for automated Let’s Encrypt certificates. So we had to stop to build the Let’s Encrypt system.
With those things completed we came back to email and realized that some of the assumptions we had made in our data model wouldn’t work with the email system correctly. So we made changes to the data model and cleaned up some cruft a long the way.
To simplify the way our system works, some configuration is created from data in the database and then turned into actual files and options on the servers. The code that creates the config we’ll call the “generator” and the code that creates the files we’ll call the “applier”.
For email there are 3 distinct systems: the MX servers, the IMAP (mailbox storage) servers, and the SMTP servers. Each of these servers requires different configuration so we had to build numerous generators and appliers, which we expected to do. Each round of updates got us closer to a correct configuration.
We also had to compile and build all of the tooling we use to manage the email system. Each update to the mail system’s underlying functionality required changes in the tooling. Plus updating the documentation for both internal operation and for all of you.
The things we didn’t expect were: the complexity of the interplay between our configuration management system, our appliers, and the version of Postfix available on CentOS 7 (v2.10 which is from 2013 + patches). In order for email to be encrypted at all points through our system we had to upgrade to a much newer version of Postfix.
With all that out of the way we ran our tests of the system successfully and wired up the automation to take care of running the generators and appliers. Which brings us to the week of July 15th – July 21st. With our final testing completed and the system functioning we set a launch date of the 23rd, in the US. With European email (in Germany) being launched 2 to 3 weeks after the US release.
As we were capacity planning (July 21st) we realized that about 45% of our customer base was based in Europe. Which would mean that everyone from Europe who started using the system in the mean time would need to migrate within a month. That would not be a good experience for you. So we decided that email in Europe and the US would have to launch at exactly the same time (July 23rd). We also made the decision to not implement email in the Asia/Pacific (Singapore) region now.
This decision meant that we had to order all of the servers to run the email system, then set them all up, add our configuration, and then test the changes we made in order to make the system regional. That work started on July 26th and finished on July 31st.
The regional DNS system we built made this both more complicated and simpler to implement than we hoped. One of the complications was that for domains without any records (like the opalstacked.com one we provide you) no MX records were created. In order to fix the assumptions we made we had to change the core data model of both the generators and appliers for all DNS records.
With that work finished on August 3rd we planned to launch today!