Introduction to SPF

SPF, or Sender Policy Framework, is an email authentication mechanism that allows domain owners to define which mail servers are authorized to send on behalf of the domain.

The list of authorized servers is specified through a DNS record of type TXT. The details of how SPF works and the syntax of the DNS record are defined in RFC 7208, the official specification of SPF.

Here’s an example of an SPF TXT record of a domain using Google Workspace to send email:

v=spf1 include:_spf.google.com ~all

An important thing to keep in mind is that SPF uses the so-called Envelope From domain to look for SPF records. This is different from the From domain, which is the one we typically see in email clients.

We’ll go through this fundamental aspect in a moment.

High-level mechanism

Let’s start with how SPF works at a high level:

• When a mail server receives a new email message, it first extracts the domain from the Envelope From address.
• It then runs a DNS query for this domain, asking for TXT records.
• The list of returned records is parsed to find if there’s an SPF record.
• The SPF servers list is “flattened”, meaning that if the record mentions other domain names (through the include directive) they’re queried to obtain a list of IP addresses (usually ranges/subnets of IP addresses).
• The mail server can finally check whether the sender IP is included in the list of IPs obtained through the steps above.
• The result of the SPF evaluation can be either PASS in case of success, or a failure. The failure type depends on the qualifiers specified in the record. These are the two most common failure qualifiers:
  • FAIL, specified by the - symbol.
  • SOFTFAIL specified by ~ (recommended).

Note that this is a simplified explanation of how SPF works: the real algorithm is a bit more complex, but the steps above conceptually represent the process.

Difference between “Envelope From” and “From”

Conceptually, SPF seems quite easy to understand: there’s a list of allowed IP addresses associated with your domain, and mail servers use that list to check whether a sender IP is authorized.

The confusing part may be that the domain used to look up the SPF record is not the domain of the sender of the message.

In fact, when sending an email through the SMTP protocol there are actually two email addresses for the sender:

• The Envelope From, used for SMTP communications between mail servers and for non-delivery notifications (bounces).
• The From header, contained in the email message. This is the one we see in email clients as the sender of a message.

SPF uses the first one, an address that is almost always invisible to users.

You might wonder why the Envelope From even exists.

If you think about it, it’s very similar to how mail works in the real world: the sender and recipient printed on a physical envelope are unrelated to the letter contained in the envelope.

For example, your electricity bill may be sent from a company specialized in sending bills on behalf of energy providers, so the sender of the envelope doesn’t match the sender of the letter contained in it. In the same way, the recipient written on the letter may be different from the place where the envelope should be delivered.

Internet mail follows the same principle: the job of email servers is to move around email messages and for this they use the information on the envelope (the delivery instructions, if you want). Mail servers don’t really need to know or care about what’s written inside the message.

This design is what makes some email features possible: an example is email forwarding, where you could have an email address that is actually an alias of another address. In this context, it wouldn’t make sense to alter the email message when forwarding it: what needs to change is the information in the SMTP envelope, which instructs mail servers where to deliver the message.

Understanding this difference is crucial to configure SPF so that it’s compliant in terms of DMARC alignment.

SPF, spoofing and DMARC

For the fact that SPF checks are performed on the Envelope From domain, SPF cannot be used to prevent email spoofing.

In fact, anyone can very easily create an email message with a fake sender (e.g. impersonating the domain of another organization) while using an Envelope From address they control. SPF checks will succeed, even if the email message is fake and the From address is spoofed.

This is where DMARC comes into play: in the context of SPF, DMARC checks whether the From domain is aligned with the SPF domain (the Envelope From domain).

This rule effectively enforces that SPF is checked on the From domain, ensuring that only mail servers that are explicitly authorized in the SPF record of the From domain are allowed to send email from that domain.

The key step to achieve SPF alignment is to use an Envelope From domain that is aligned with your organizational domain.

For example, if you own the domain dmarcwise.io you should make sure that all your sending software uses that domain (or a subdomain) in the Envelope From.

Depending on the email software or service that you use you might find this feature called custom Return-Path or custom MAIL FROM. In some cases, email providers don’t allow using an aligned Envelope From at all.

Limitations

DNS lookup limit

As mentioned above, the SPF record may contain references to other domains, for example in the include directive. These domains need to be resolved to map them to a list of IP addresses. Additionally, each domain name may itself require the resolution of other domain names.

The SPF specification says that SPF implementations must limit the total number of DNS queries run during an SPF check to 10.

In complex email setups it isn’t infrequent to reach this limit, so it needs to be monitored to prevent avoidable SPF failures.

Automatic forwarding and SPF failures

When automatic forwarding/relaying is enabled for a mailbox (keep in mind that this is unrelated to the “forward” button in your email client), by default the email message is sent again by the mail server with the same configuration as the original.

This means that not only the From address is left unaltered, but also the Envelope From. When the email reaches the destination, SPF checks will likely fail because the IP address of the forwarding mail server is not authorized by the SPF record.

The solution to this problem is a feature called Sender Rewriting Scheme (SRS). With SRS, a forwarding mail server rewrites the Envelope From such that the forwarding mail server is authorized by the SPF record of the “new” domain.

SRS also takes care of making sure that non-delivery notifications, or bounces, are delivered to the original address as intended. This is done by encoding the original Envelope From address in the new one with a special format.

Automatic forwarding and alignment

When Sender Rewriting Scheme is enabled (as it is very often), the Envelope From is rewritten to a new email address and domain under the control of the forwarding server.

Although this will make SPF pass, it will break alignment with the From domain, which remains unchanged.

This is a very common issue when deploying DMARC and may sometimes be mitigated by a relatively new mechanism called ARC (Authenticated Received Chain), which “freezes” the authentication results before forwarding. The destination mail server would then be able to trust the ARC information and recover from alignment failures.