MTA-STS in Enforce Mode: How a Single Misconfigured Policy Blocks Every Gmail Message

What MTA-STS Actually Does to Your Mail Flow

MTA-STS (Mail Transfer Agent Strict Transport Security) forces sending mail servers to verify TLS encryption before delivering a message. Without it, SMTP connections between servers default to opportunistic encryption, meaning a man-in-the-middle can downgrade the connection to plaintext without either side noticing.

The protocol works through three components: a DNS TXT record at _mta-sts.yourdomain.com, a policy file hosted at https://mta-sts.yourdomain.com/.well-known/mta-sts.txt, and an optional TLSRPT record for receiving daily failure reports. When Gmail sends an email to your domain, it checks the DNS TXT record first. If the record exists, Gmail fetches the policy file over HTTPS, reads the mode, validates the listed MX hosts, and caches the result for however many seconds max_age specifies. A policy set to enforce mode tells Gmail: refuse delivery if TLS negotiation fails, if the certificate is invalid, or if the receiving MX hostname does not match the policy file.

That last part is where things break.

The Policy File That Stops Delivery

Here is a typical MTA-STS policy file:

version: STSv1
mode: enforce
mx: mail.yourdomain.com
mx: backup.yourdomain.com
max_age: 604800

Every MX entry in this file must match the actual MX DNS records for your domain. If your DNS has mx: mail.yourdomain.com but your policy file lists mx: smtp.yourdomain.com, Gmail will reject the connection. The certificate presented by your mail server must also match the hostname, be issued by a trusted root CA, and support TLS 1.2 or higher. Expired certificates, self-signed certificates, and hostname mismatches all trigger the same result: bounced email. Gmail returns a 4xx or 5xx error, retries for a few days, and eventually generates a bounce notification to the sender.

This behavior is identical for Google Workspace accounts sending outbound mail. Gmail honors MTA-STS policies published by recipient domains, so a misconfigured policy on the receiving end blocks delivery from every MTA-STS-capable sender, not just Gmail.

Why Firewalls and SMTP Proxies Cause the Most Failures

Many network appliances inspect SMTP traffic by inserting a proxy between your mail server and the outside world. Firewalls from vendors like SonicWall, Fortinet, and Sophos often include an SMTP filtering module that terminates the TLS connection, inspects the payload, then re-encrypts it using the firewall’s own certificate. Under normal SMTP, this works without complaint. Under MTA-STS enforce mode, it breaks everything.

Gmail expects to see a valid certificate matching your MX hostname. The firewall presents its own certificate instead, which does not match. Gmail treats this as a certificate validation failure and refuses to deliver. The security layer between your server and external connections must be configured to pass TLS through without interception for port 25 traffic. Disabling the SMTP inspection module on your firewall is the fastest fix. Alternatively, configure the firewall to bypass inspection for connections originating from known Gmail IP ranges.

The diagnostic step is straightforward. From a machine on the same network as your mail server, run:

openssl s_client -connect mail.yourdomain.com:25 -starttls smtp

Check the certificate chain in the output. If the issuer is your firewall appliance instead of a public CA like Let’s Encrypt or DigiCert, that is the problem.

The max_age Trap: Cached Policies That Outlive Your Fix

Fixing the policy file does not immediately fix delivery.

Gmail caches the MTA-STS policy for the duration specified in max_age. A value of 604800 means seven days. If you published a broken policy with max_age: 2592000 (30 days), Gmail will enforce that broken policy for up to 30 days even after you correct the file. The DNS TXT record includes an id field that signals policy changes, but Gmail only checks this periodically, not on every delivery attempt. Changing the id value (for example, from 20260301000000 to 20260315000000) tells Gmail to re-fetch the policy, but re-fetch timing depends on Gmail’s internal schedule and the remaining TTL of the cached entry.

The only reliable way to stop the bleeding during an emergency is to change the mode from enforce to none in the policy file, update the id in the DNS TXT record, and wait. There is no manual cache flush available on the Gmail side. Some administrators overlook how DNS propagation delays compound this problem. If your DNS TTL is set high, the updated TXT record itself takes hours to propagate.

Setting Up MTA-STS the Right Way on a WordPress Server

Most WordPress sites run on shared or VPS hosting where the mail server is either Postfix, Exim, or a third-party service like Google Workspace or Zoho. The MTA-STS policy file must be served over HTTPS from a specific subdomain, which means you need to create a DNS A record for mta-sts.yourdomain.com, point it to a web server, install a valid TLS certificate on that subdomain, and host the policy file at the exact path /.well-known/mta-sts.txt.

On an Nginx server, the configuration looks like this:

server {
    listen 443 ssl;
    server_name mta-sts.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/mta-sts.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mta-sts.yourdomain.com/privkey.pem;

    location /.well-known/mta-sts.txt {
        root /var/www/mta-sts;
        default_type text/plain;
    }
}

Create the policy file at /var/www/mta-sts/.well-known/mta-sts.txt with your actual MX hostnames. If you use Google Workspace, the MX entries should reference *.aspmx.l.google.com or the specific ASPMX hostnames listed in your Workspace DNS configuration. Hosting your web traffic on HTTP/3 does not affect the MTA-STS subdomain, which only needs standard HTTPS.

The DNS records you need:

_mta-sts.yourdomain.com  TXT  "v=STSv1; id=20260326100000"
_smtp._tls.yourdomain.com  TXT  "v=TLSRPTv1; rua=mailto:[email protected]"

The second record enables TLS Reporting, which sends you daily JSON reports from every sending MTA that attempted delivery. These reports list successful sessions, failed sessions, failure reasons (certificate-expired, certificate-host-mismatch, sts-policy-fetch-error), and the sending server’s IP address.

Reading TLS Reports to Catch Problems Before Enforce Mode

Start with mode: testing and leave it there for at least two weeks.

In testing mode, Gmail still delivers mail even when TLS validation fails, but it sends a report to the address in your TLSRPT record. The report is a gzipped JSON file containing a policies array, a summary of successful and failed sessions, and a failure-details array with specific error types. The result-type field tells you exactly what went wrong: starttls-not-supported, certificate-expired, certificate-host-mismatch, or sts-policy-invalid. Cross-reference the sending-mta-ip with your server access logs to identify which senders hit failures. Fix every reported issue before switching to enforce mode. A clean two-week run with zero failures in the reports means your infrastructure is ready.

Switching from testing to enforce requires editing the policy file and updating the id in the DNS TXT record. Both changes must happen together, because the id change is what triggers remote MTAs to re-fetch the policy.

Common Mistakes That Block Gmail Delivery

A wildcard MX entry in the policy file (mx: .yourdomain.com) only works if your MX DNS records also use a pattern that matches. If your MX record points to mail.yourdomain.com but your policy says mx: .yourdomain.com, that is valid. If your MX record points to mail.hosting-provider.com, the wildcard *.yourdomain.com does not match, and delivery fails under enforce mode.

Certificate auto-renewal failures are another frequent cause. Let’s Encrypt certificates expire every 90 days. If the cron job handling renewal breaks, your MX server starts presenting an expired certificate within three months. Gmail will not deliver to a server with an expired cert when MTA-STS enforce mode is active. Automating certificate renewal checks through a monitoring service (Uptime Robot, Healthchecks.io) that alerts on expiry is the minimum viable safeguard.

Forgetting to update the id field after editing the policy file is the third classic mistake. The id acts as a version identifier. Remote MTAs compare the id in DNS with their cached copy. If the id has not changed, they skip re-fetching the policy, meaning your fix sits on the server doing nothing. Use a timestamp-based id format like YYYYMMDDHHMMSS to make versioning automatic.

Validating Your MTA-STS Setup

Run through this checklist before switching to enforce:

# Fetch the policy file
curl -v https://mta-sts.yourdomain.com/.well-known/mta-sts.txt

# Check the DNS TXT record
dig TXT _mta-sts.yourdomain.com +short

# Verify TLS on your MX server
openssl s_client -connect mail.yourdomain.com:25 -starttls smtp

# Check TLSRPT record
dig TXT _smtp._tls.yourdomain.com +short

The curl command should return a 200 status with the correct policy content. The TLS check should show a certificate chain from a public CA with a matching hostname and TLS 1.2 or higher. If your site already forces HTTPS, the same certificate management practices apply to the MTA-STS subdomain. Google Workspace administrators can also validate their configuration directly in the Admin console under Apps > Google Workspace > Gmail > Advanced settings, where a built-in diagnostic tool checks MTA-STS records and suggests corrections.

Online validators like MXToolbox’s MTA-STS checker, Mailhardener’s inspector, and dmarcian’s MTA-STS tool all perform automated checks against your DNS records, policy file accessibility, certificate validity, and MX hostname matching. Running at least two different tools catches edge cases that a single validator might miss.

Често задавани въпроси