Back to tutorials
Tutorial

Postfix mail server setup tutorial (2026): Build a small-business SMTP relay on a VPS with TLS, SPF/DKIM, and smart rate limits

Postfix mail server setup tutorial for 2026: configure SMTP relay on a VPS with TLS, SPF/DKIM, rDNS, and safe rate limits.

By Anurag Singh
Updated on Sep 27, 2026
Category: Tutorial
Share article
Postfix mail server setup tutorial (2026): Build a small-business SMTP relay on a VPS with TLS, SPF/DKIM, and smart rate limits

Email problems usually aren’t “random.” They usually come from a few basics: a mismatched hostname, missing reverse DNS, broken TLS, or DNS records that don’t line up.

This Postfix mail server setup tutorial shows how to build a practical SMTP relay on a Linux VPS in 2026. It works well for web apps, transactional email, and small-business outbound. It also shows how to avoid creating an open relay.

This guide covers outbound mail only. You’ll set up submission on 587/465, authenticated sending, TLS, SPF/DKIM/DMARC, and sensible throttling.

Inbound mailbox hosting (IMAP/POP) is out of scope. That’s where the operational burden and abuse risk spike.

What you’ll build (and what you should not build)

You’ll set up a relay that accepts mail from your apps and users. It then delivers to the internet using your VPS IP reputation and clean DNS alignment.

The server will:

  • Listen on 587 (STARTTLS) and optionally 465 (SMTPS)
  • Require SASL authentication for sending
  • Use TLS certificates that renew automatically
  • Sign outbound mail with DKIM
  • Publish correct SPF and DMARC records
  • Apply rate limits to slow abuse without blocking your own app bursts

What you should not build (for most small teams): a full mail platform with inbound mailboxes, webmail, spam filtering, and multi-tenant user management.

Don’t do it unless you’re ready to maintain it long-term.

Prerequisites checklist (do these first)

Before you install anything, lock down the identity of the mail server. If you skip this, you’ll waste time chasing “deliverability” issues that are really alignment problems.

  • A VPS with a dedicated IPv4 (recommended). Pick a plan with room for bursts and logs. A HostMyCode VPS is a straightforward fit for an SMTP relay.
  • A domain you control (example: example.com)
  • A subdomain for mail identity (example: mail.example.com)
  • Access to DNS management for the domain (or a registrar DNS panel)
  • Ubuntu 24.04 LTS or Debian 12 (commands below use Ubuntu 24.04; Debian is almost identical)

Decide how your applications will submit mail:

  • Option A (recommended): your app authenticates to the relay on 587 with a username/password
  • Option B: your app runs on the same VPS and submits via localhost (still keep auth for remote clients)

If this is your first time hardening a new server, start with a safe baseline here: secure a new Ubuntu VPS for hosting without lockouts.

Step 1: Set the hostname and verify reverse DNS

Your SMTP greeting (HELO/EHLO) must match a real, resolvable name. Your IP’s PTR (reverse DNS) should point to that same name.

Many receivers downgrade or reject mail when these values don’t match.

1) Set your system hostname to the mail FQDN:

sudo hostnamectl set-hostname mail.example.com
hostnamectl

2) Ensure the hostname resolves to your server IP (A/AAAA record). You’ll set DNS shortly.

For now, check what’s visible:

dig +short mail.example.com A
dig +short mail.example.com AAAA

3) Configure PTR / rDNS with your provider so your IP points to mail.example.com. Then verify it:

dig +short -x YOUR.SERVER.IP.ADDRESS

Need a tight, testable workflow for PTR? Use: configure reverse DNS and verify mail & hostname.

Step 2: Open only the ports you actually need

For an outbound relay, you typically need:

  • 22/tcp (SSH) from your admin IPs
  • 587/tcp (submission) for authenticated clients
  • 465/tcp (optional) for legacy clients that require SMTPS
  • 25/tcp (SMTP) outbound to deliver to other mail servers; inbound 25 may be required for delivery and reputation checks, but you can restrict who can submit mail so 25 isn’t a “send port” for users

If you’re using UFW on Ubuntu:

sudo ufw allow OpenSSH
sudo ufw allow 587/tcp
sudo ufw allow 25/tcp
sudo ufw allow 465/tcp
sudo ufw enable
sudo ufw status verbose

For a conservative firewall baseline (and to avoid locking yourself out), see: UFW + cloud firewall + safe SSH rules.

Step 3: Install Postfix and SASL (Dovecot auth) on Ubuntu

Postfix handles SMTP reliably. Dovecot’s SASL components provide a familiar, well-supported authentication layer.

sudo apt update
sudo apt -y install postfix postfix-pcre dovecot-core dovecot-imapd ca-certificates

During the Postfix prompt:

  • Select Internet Site
  • System mail name: example.com (or mail.example.com; we’ll set explicit values in config next)

Confirm Postfix is running:

systemctl status postfix --no-pager

Step 4: Configure Postfix for authenticated submission (587) and safe defaults

Edit /etc/postfix/main.cf with a light touch. Don’t paste a giant template you won’t recognize later.

Make changes you can explain and roll back.

sudo postconf -n

Set or adjust these values (use sudo postconf -e to avoid typos):

sudo postconf -e 'myhostname = mail.example.com'
sudo postconf -e 'mydomain = example.com'
sudo postconf -e 'myorigin = $mydomain'
sudo postconf -e 'inet_interfaces = all'
sudo postconf -e 'inet_protocols = all'

# Don’t accept mail for random domains
sudo postconf -e 'mydestination = localhost'

# Relay restrictions: allow authenticated, allow local, reject everything else
sudo postconf -e 'smtpd_relay_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination'

# Networks allowed without auth (keep tight)
sudo postconf -e 'mynetworks = 127.0.0.0/8 [::1]/128'

# Basic anti-abuse posture
sudo postconf -e 'smtpd_helo_required = yes'
sudo postconf -e 'disable_vrfy_command = yes'
sudo postconf -e 'smtpd_delay_reject = yes'

Why mydestination = localhost? It prevents Postfix from accepting mail “for” example.com as if you host inbound mailboxes.

For a relay-only server, that’s usually the right behavior.

Now configure the submission service in /etc/postfix/master.cf. On Ubuntu, the submission entries often exist but are commented out.

Make sure you have a submission service like this:

sudo nano /etc/postfix/master.cf
submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject

This is the core safety model. Port 587 requires TLS and authentication.

Postfix also rejects unauthenticated recipients. That’s what keeps the server from becoming an open relay.

If you need port 465, add:

smtps     inet  n       -       y       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject

Step 5: Configure Dovecot SASL for Postfix

Here, Dovecot only provides auth (SASL). You are not turning this server into an IMAP host.

That keeps the footprint smaller and the attack surface tighter.

Edit /etc/dovecot/conf.d/10-master.conf and ensure the auth socket exists for Postfix:

sudo nano /etc/dovecot/conf.d/10-master.conf

Find the service auth block and add/enable:

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

Restart services:

sudo systemctl restart dovecot
sudo systemctl restart postfix

Create a dedicated SMTP user (don’t reuse root):

sudo adduser smtpuser
sudo passwd smtpuser

Later, your app will use smtpuser as the SMTP username.

Step 6: Add TLS certificates (Let’s Encrypt) and wire them into Postfix

Receivers expect modern TLS. Your clients should also never submit mail in plaintext.

Install Certbot and issue a certificate for mail.example.com:

sudo apt -y install certbot
sudo certbot certonly --standalone -d mail.example.com

Cert files:

  • /etc/letsencrypt/live/mail.example.com/fullchain.pem
  • /etc/letsencrypt/live/mail.example.com/privkey.pem

Configure Postfix TLS in main.cf:

sudo postconf -e 'smtpd_tls_cert_file=/etc/letsencrypt/live/mail.example.com/fullchain.pem'
sudo postconf -e 'smtpd_tls_key_file=/etc/letsencrypt/live/mail.example.com/privkey.pem'
sudo postconf -e 'smtpd_tls_security_level=may'
sudo postconf -e 'smtpd_tls_loglevel=1'
sudo postconf -e 'smtpd_tls_received_header=yes'

# For outbound (Postfix as SMTP client)
sudo postconf -e 'smtp_tls_security_level=may'
sudo postconf -e 'smtp_tls_loglevel=1'

Restart Postfix:

sudo systemctl restart postfix

If renewals ever break (stale paths, permission issues, failed standalone binds), keep this guide bookmarked: fix auto-renew failures on common stacks.

Step 7: Publish DNS records (A, MX, SPF, DKIM, DMARC)

Your relay can send without an MX record. Still, publishing one for example.com that points to mail.example.com makes the setup look consistent to receivers.

At minimum, publish A, SPF, DKIM, and DMARC.

7.1 A (and optional AAAA)

  • A: mail.example.com → your IPv4
  • AAAA (optional): mail.example.com → your IPv6 (only if you’ve configured IPv6 correctly)

7.2 MX

For a typical domain:

  • MX for example.com → mail.example.com priority 10

7.3 SPF

Start strict, but leave room for growth. If this VPS is the only sender for example.com:

example.com.  TXT  "v=spf1 a:mail.example.com -all"

If you also send from a third-party provider (like a newsletter tool), include their SPF mechanism too.

Don’t publish multiple SPF TXT records. Merge everything into one.

7.4 DKIM

We’ll generate a DKIM key with OpenDKIM in the next step. Then you’ll add a DNS TXT record like:

default._domainkey.example.com. TXT "v=DKIM1; k=rsa; p=..."

7.5 DMARC

Start in monitoring mode. Tighten later:

_dmarc.example.com. TXT "v=DMARC1; p=none; rua=mailto:dmarc-reports@example.com; fo=1"

Once alignment is stable, move to p=quarantine or p=reject.

If you tighten DMARC before DKIM/SPF alignment is solid, you can break legitimate mail quickly.

If you want a structured, receiver-friendly setup (including rDNS and TLS expectations), this HostMyCode guide pairs well with this tutorial: configure SPF, DKIM, DMARC, rDNS, and TLS.

Step 8: Install and configure OpenDKIM to sign outbound mail

DKIM signing is often the difference between “it delivered” and “it landed in Inbox.” Without DKIM, you ask receivers to trust you on IP reputation alone.

sudo apt -y install opendkim opendkim-tools

Create directories:

sudo mkdir -p /etc/opendkim/keys/example.com
sudo chown -R opendkim:opendkim /etc/opendkim
sudo chmod 750 /etc/opendkim/keys

Generate a key (selector: default):

sudo opendkim-genkey -b 2048 -d example.com -D /etc/opendkim/keys/example.com -s default
sudo chown opendkim:opendkim /etc/opendkim/keys/example.com/default.private
sudo chmod 600 /etc/opendkim/keys/example.com/default.private

Create OpenDKIM config files:

sudo nano /etc/opendkim.conf
Syslog                  yes
UMask                   002
Canonicalization        relaxed/simple
Mode                    sv
SubDomains              no
AutoRestart             yes
AutoRestartRate         10/1h
OversignHeaders         From
Socket                  local:/run/opendkim/opendkim.sock
UserID                  opendkim

Key tables:

sudo nano /etc/opendkim/key.table
default._domainkey.example.com example.com:default:/etc/opendkim/keys/example.com/default.private
sudo nano /etc/opendkim/signing.table
*@example.com default._domainkey.example.com
sudo nano /etc/opendkim/trusted.hosts
127.0.0.1
::1
localhost
mail.example.com

Ensure OpenDKIM socket directory exists and is accessible:

sudo mkdir -p /run/opendkim
sudo chown opendkim:opendkim /run/opendkim

Now connect Postfix to OpenDKIM by adding these to main.cf:

sudo postconf -e 'milter_default_action=accept'
sudo postconf -e 'milter_protocol=6'
sudo postconf -e 'smtpd_milters=unix:/run/opendkim/opendkim.sock'
sudo postconf -e 'non_smtpd_milters=unix:/run/opendkim/opendkim.sock'

Restart services:

sudo systemctl restart opendkim
sudo systemctl restart postfix

Publish DKIM DNS record: output the public key:

sudo cat /etc/opendkim/keys/example.com/default.txt

Copy the TXT value into your DNS provider. Keep it as one logical string (some panels split it automatically).

Step 9: Add smart rate limits to reduce abuse (without breaking real traffic)

Rate limiting is your seatbelt. It won’t stop a fully compromised account.

It does reduce blast radius and buys you time before your IP reputation tanks.

In main.cf, set conservative limits for a small-business relay:

sudo postconf -e 'smtpd_client_message_rate_limit=200'
sudo postconf -e 'smtpd_client_recipient_rate_limit=400'
sudo postconf -e 'smtpd_client_connection_rate_limit=60'
sudo postconf -e 'smtpd_error_sleep_time=1s'
sudo postconf -e 'smtpd_soft_error_limit=10'
sudo postconf -e 'smtpd_hard_error_limit=20'

Restart Postfix:

sudo systemctl restart postfix

How to tune:

  • If WooCommerce order bursts cause temporary deferrals, raise smtpd_client_message_rate_limit.
  • If you see credential stuffing, lower connection rate and add a firewall-level blocklist.
  • If one app sends high volume, consider a dedicated SMTP user for it, so you can cut it off quickly.

Step 10: Test submission (587) from your laptop and from the server

Before you call this “done,” verify three things: TLS negotiation, authentication, and delivered headers that show authentication results.

10.1 Confirm TLS and the certificate

openssl s_client -starttls smtp -connect mail.example.com:587 -servername mail.example.com

You’re looking for a valid certificate chain and a clean handshake.

10.2 Send a test mail with swaks

swaks is the fastest way to test SMTP auth and TLS. It also helps you avoid guessing what your app is doing.

sudo apt -y install swaks
swaks --to you@gmail.com \
  --from test@example.com \
  --server mail.example.com \
  --port 587 \
  --auth LOGIN \
  --auth-user smtpuser \
  --auth-password 'YOUR_PASSWORD' \
  --tls

Check the received message headers in the destination mailbox. You should see:

  • DKIM-Signature present
  • SPF=pass for your sending IP
  • DMARC either pass or at least aligned (depends on policy)

If mail lands in spam, don’t guess. Work through alignment, rDNS, and queue behavior using: fix VPS mail going to spam with SPF/DKIM alignment and queue checks.

Step 11: Watch logs like an operator (and catch problems early)

Mail problems show up in logs before they become support tickets. On Ubuntu, Postfix logs to /var/log/mail.log.

Tail logs during tests:

sudo tail -f /var/log/mail.log

Useful quick commands:

  • See current queue: mailq
  • Flush queue: sudo postfix flush
  • Requeue deferred mail: sudo postfix -f

If you need to trace one message end-to-end (queue ID and remote response codes), use: trace SMTP delivery failures using logs, headers, and queue IDs.

Step 12: Backup the “mail identity” pieces (so rebuilds don’t break trust)

For an outbound relay, the critical backups are small. They’re also the items that can quietly wreck trust and deliverability if you lose them.

  • /etc/postfix/
  • /etc/opendkim/ (especially default.private)
  • Let’s Encrypt: /etc/letsencrypt/
  • Your DNS zone records (export if your DNS provider supports it)

If you rotate DKIM keys, do it deliberately. Keep overlap time so older messages still validate.

Also make sure this relay is part of your normal VPS backup plan. If you only prioritize one item, prioritize DKIM private keys.

Operational checklist: common mistakes that break deliverability

  • PTR mismatch: IP reverse DNS points somewhere else, or to a generic provider name.
  • HELO mismatch: server says mail.example.com but DNS doesn’t resolve.
  • SPF is too strict: you forgot another legitimate sender, so receivers see SPF fail.
  • DKIM not signing: OpenDKIM socket path mismatch or wrong permissions on the private key.
  • Submission allows unauthenticated mail: missing permit_sasl_authenticated and reject in master.cf submission restrictions.
  • Firewall blocks 25 outbound: queue grows, mail defers, and apps time out.

Summary: a relay you can trust (and troubleshoot)

You now have a clean relay posture: authenticated submission, TLS, DKIM signing, and DNS alignment that receivers can verify.

Just as important, the setup is debuggable.

When something fails, you have three places to look: DNS, logs, and the queue. Each one gives concrete signals you can act on.

If you want a stable platform for outbound mail (and enough headroom to handle bursts safely), start with a HostMyCode VPS.

If you’d rather not babysit patching, firewall rules, and service restarts, consider managed VPS hosting and keep your focus on your applications.

A reliable SMTP relay comes down to consistency: a stable IP, correct rDNS, and predictable behavior under load. HostMyCode can provision the right foundation on a HostMyCode VPS, or handle ongoing maintenance with managed VPS hosting so your mail keeps landing where it should.

FAQ

Do I need to open port 25 inbound for an outbound relay?

For pure submission, your clients use 587/465. Port 25 is still used for server-to-server delivery, and some receivers may test connectivity. Keep 25 open, but don’t let users submit mail through it.

Should my relay have an MX record if I’m not receiving inbound mail?

It’s optional. Publishing an MX that points to mail.example.com can make your domain’s mail identity consistent. Just don’t create inbound mailboxes unless you intend to operate them.

What’s the fastest way to confirm DKIM is working?

Send a test to a mailbox that shows authentication results. Then check headers for DKIM-Signature and a dkim=pass result. If it’s failing, check permissions and the milter socket path first.

My emails send, but they land in spam. What should I fix first?

Start with rDNS/PTR and hostname alignment. Next, confirm SPF/DKIM alignment and your DMARC policy. After that, check queue deferrals and remote SMTP response codes to see what receivers are rejecting.

Can I use this relay for WordPress sites hosted elsewhere?

Yes. Configure WordPress (or your SMTP plugin) to authenticate to mail.example.com:587 with TLS. Use separate SMTP users per site so you can isolate abuse or misconfigurations quickly.