One Binary for All of Email
Email is the service this whole project exists for, and the one my family will not forgive me for breaking. It’s also the service with the scariest reputation in self-hosting. Part of that reputation comes from the traditional stack: Postfix for SMTP, Dovecot for IMAP, rspamd for spam, something else for CalDAV, each with its own config language, its own failure modes, and its own opinions about the others.
I went a different way. Stalwart is a single Rust binary that speaks SMTP, IMAP, JMAP, CalDAV, and CardDAV. One process, one configuration, one thing to understand deeply instead of five things to understand adequately. It’s younger than Postfix by decades, which is a real risk I accepted with open eyes. This post is about getting it running; convincing the rest of the internet to accept its mail is a story of its own, coming next week.
Push all the state out
The design decision that shaped everything else: the mail nodes themselves hold nothing irreplaceable. Stalwart supports pluggable backends, so every kind of state went to a managed service. Mail data and accounts in PostgreSQL. Caching and cluster coordination in Valkey. Full-text search in OpenSearch. Message blobs in S3-compatible object storage. All four are Exoscale managed services with their own replication and backups.
What’s left on the node is a container, a config file rendered by Ansible, and TLS certificates. That makes mx1 and mx2 disposable in the best sense. Rebuilding one is a Terraform apply and an Ansible playbook, not a data recovery operation.
Active-active, not primary-standby
Both nodes run simultaneously, in different data centers (Frankfurt and Zurich), against the same backends. The configuration is identical except for a node ID. I expected cluster coordination to be the hard part; Stalwart coordinates through Valkey pub/sub and it worked on the first deploy. I have almost nothing to report, which for clustering software is the highest compliment I know.
DNS round-robin spreads client connections across both nodes, and mail protocols were built in an era of unreliable everything, so clients retry the other node when one disappears. I’ve shut nodes down deliberately to watch this: mail keeps flowing, phones keep syncing, nobody notices.
Breaking it on purpose
Before calling the epic done I spent an evening cutting each backend off with firewall rules and watching what Stalwart did about it. PostgreSQL gone for fifty seconds: connection pool rides it out, sessions continue, not a single ERROR-level log line. Same for Valkey and OpenSearch. When the backend comes back, reconnection is automatic. The only user-visible symptom during the outage windows was search feeling slightly stale.
I recommend this exercise for any service you intend to depend on. The failure modes you trigger yourself, on a calm evening with coffee, are the ones that won’t surprise you at 2 AM. It also settled a design worry: with all state shared, a flaky backend degrades both nodes equally, but it degrades them gracefully.
The one deployment problem worth reporting had nothing to do with Stalwart: mid-playbook, unattended-upgrades grabbed the apt lock on mx2 and sat on it, and I got to practice the ancient art of SSHing in to kill a process. The machines remain undefeated at finding new ways to be machines.
The port nobody tells you about
The listening ports are the classics: 25 for inbound SMTP, 465 for authenticated submission, 993 for IMAPS. Then in June I added a second submission listener on port 2525, identical to 465 in every way: implicit TLS, authentication required, DKIM signing on the way out.
The reason is mundane and infuriating. Some hotel and office networks block outbound 465 wholesale, and when that happens to someone in my family, “my mail won’t send” becomes a firewall I can’t debug from a distance. So the same service got a second door on a port those filters tend to ignore. Relay tests confirmed it refuses unauthenticated mail exactly like its twin, and I verified a real send over 2525 myself before calling it done. Most of the effort was that verification.
Next week: SPF, DKIM, DMARC, MTA-STS, and the delicate diplomacy of getting Gmail to trust a two-node operation in Frankfurt.
You May Also Like
LUKS on Every Data Volume
Full-disk encryption for servers I will never physically see, and why the unlock step stays manual.
Secrets That Live in Git
How SOPS and two GPG keys let me commit every credential this platform needs to the repository itself.
Catching Up: Six Months of Building, Zero Posts
The blog went quiet in February while the platform kept growing. Here's what happened, and the plan for writing it all down.
