The Holy Trinity of Email Verification - SPF, DKIM, and DMARC

- 12 mins
research

Many blogs and write-ups describe DMARC as an additional security measure that builds upon SPF and DKIM for anti-spoofing and prevent phishing, and then attempt to pitch their own DMARC analyzer tool to provide “advanced malware protection” against attackers.

Yes I’m looking at you, Mimecast.

The issue with these blogs is that they don’t really explain what’s going on behind the scenes. They throw together some terms like “alignment” and “verification”. But what exactly is “alignment”? How is an email message “verified”? And why does DMARC fail even when both SPF and DKIM pass?

The goal here is to provide a technical overview of how SPF, DKIM, and DMARC work without having to deal with the BS of vendors forcing their product into your faces.

DMARC in a nutshell

DMARC stands for Domain-based Message Authentication, Reporting, and Conformance. DMARC uses SPF and DKIM to perform validation on incoming email messages. Depending on the output of SPF and DKIM checks, DMARC will either accept or reject incoming emails.

DMARC records can be found in a TXT record for the _dmarc subdomain of a given domain. For example, if I wanted to get the DMARC record for steampowered.com, I would query for a TXT record at _dmarc.steampowered.com:

v=DMARC1; p=reject; rua=mailto:e8b9a9244a5c591@rep.dmarcanalyzer.com; ruf=mailto:e8b9a9244a5c591@for.dmarcanalyzer.com; pct=100; sp=reject; fo=1;

Here’s what each of the items mean in the above DMARC record:

The most important item here is the p field, which stands for “policy”. It’s the action to take whenever DMARC fails. The possible values for p are:

A missing DMARC record or a DMARC policy of none can be abused to allow attackers to perform email spoofing attacks. However, if the none policy is applied, email messages that fail DMARC validation can still be tracked, and most email clients will flag the unverified email message as malicious.

Before we dive a bit deeper into DMARC, let’s first discuss about SPF and DKIM.

What is SPF?

SPF stands for Sender Policy Framework. It lays out the guidelines on who is allowed to send email from a specific domain. You can think of it like a firewall ACL for email messages.

SPF is configured in the form of a TXT record of a given domain. Here’s what the SPF record looks like for steampowered.com:

v=spf1 mx ip4:72.165.61.134/31 ip4:208.64.202.32/27 -all

And each of the items:

Each of the items in the SPF record are called “mechanisms”. So you would refer to mx as the mx mechanism, ip4 as the ip4 mechanism, and so on. In general, they’re referred to as “tags” but in SPF they’re specifically “mechanisms”.

You may see ~all or +all or ?all in other SPF records. The character behind the all are referred to as qualifiers or prefixes. Here’s what they mean:

This is how SPF works in a nutshell:

  1. An email server receives an incoming email message.
  2. The email server looks at the domain of the Return-Path header of the email message.
  3. The email server retrieves the SPF record of the Return-Path domain.
  4. If the source IP of the email message matches the IP addresses specified within the SPF record, then the email message is said to be SPF authenticated.
  5. The email server will then decide how to deliver the email based on the SPF record’s all mechanism.

A key step to point out is that SPF uses the Return-Path header, which is the email used for things like bounced messages. Why does SPF use the Return-Path header? I don’t really know.

What is DKIM?

DKIM stands for DomainKeys Identified Mail. It ensures that the sender of an email address is who they actually are, as well as that the email message has not been modified in transit.

DKIM uses digital signatures and asymmetric key encryption to achieve this. Emails are sent with a DKIM signature, which contains encrypted hashes of some of the email headers and the body of the message. The specific email headers chosen to be included in the DKIM signature are configured at the email server.

This is the DKIM record associated with steampowered.com:

v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDESbiiHOv8WD5RPZ8UAYkS+rmxWyWV7l+g5jtfkZEKsOokamC9RnRHBlZiKucmA1o0ffmg+Z8KAFjf4Yy9OL1OGEO20F3UfMACffbJNsun67J3V7XWBsJwczEsJ21rmAhF0c9ntyg6yGAxwiYfzONhi0WKuN1QLslFjBRcMLoIrQIDAQAB

In order to query the DKIM record, you would need to know the selector that is being used. The selector is a value that is included in the DKIM signature to specify the location of the DKIM record.

Therefore, the DKIM record would be located at [selector]._domainkey.<domain>.

The selector can be any arbitrary string of text. For example, if I want my selector to be deeznuts then my DKIM TXT record will be located at deeznuts._domainkey.tsnguyen.com. The sending email server then must be configured to use deeznuts as the selector of the DKIM signature.

This is how DKIM works in a nutshell:

  1. The sending email server computes the hashes of email headers and body.
  2. The sending email server encrypts the hash with a DKIM private key.
  3. Sending email server includes the encrypted hash into a DKIM signature, along with other information such as the hashing algorithm used and the domain and selector to use to query DKIM records.
  4. Email is sent to client, along with the DKIM signature (that contains the encrypted hash).
  5. The client email server sees the DKIM signature and obtains the DKIM public key from querying the DKIM record.
  6. The client email server decrypts the hash from DKIM signature using the DKIM public key.
  7. The client email server generates its own hash of the email headers and body.
  8. If the hashes are the same, the client email server verifies that the email and sender are valid, and the email is said to be DKIM authenticated.

There’s a lot more that happens underneath the hood within DKIM, but personally I wouldn’t really be too concerned about the inner workings. It’s also worth mentioning that the domain used to obtain the public key (and therefore used to validate DKIM) is based on the d= field in the DKIM-Signature email header.

Extra DKIM notes

DKIM signers usually refer to the systems within the email chain that are responsible for performing the hashing and encrypting of email messages, while DKIM verifiers usually refer to the systems on the receiving end of the email message that decrypt and compare the hashes. They are used more often describe the DKIM process rather than describing individual entities or systems.

DMARC Alignment

We’ve talked about how email messages can be SPF and DKIM authenticated. But DMARC throws in another term called alignment, which is super important because it’s actually the alignment that determines whether DMARC passes or fails.

According to RFC 7489, an authenticated identifier is a domain that has been validated using either SPF or DKIM. That would be the Return-Path value for SPF and the d= value for DKIM.

DMARC identifier alignment occurs when the domain authenticated by SPF and/or DKIM matches the From: header of the email message.

Why “and/or”? Because DMARC only requires one of the two authentication mechanisms to pass and align with the sending domain.

That means if SPF passes but DKIM fails, as long as the domain used by SPF matches with the domain in the From: field, then DMARC passes as well. Now if SPF and DKIM both pass but DMARC fails, then that means that the domain in the From: field may not match either of the domains evaluated by SPF and DKIM.

In other words, DMARC checks are validated when:

Example of SPF and DKIM passing but DMARC failing

Let’s look at the following email message:

It seems that SPF and DKIM both pass but DMARC fails. Let’s take a closer look at the email headers.

If we only look at the important headers, we can extract the following domains used for each header:

SPF uses the domain of the Return-Path header to perform the SPF check. If we look at the Return-Path of this email message, we can see that it is amazonses.com. Similarly, if we look at the domain used to validate DKIM, it is also amazonses.com, listed as the d= field in the DKIM-Signature header.

However, our From address is a gmail.com domain. Because gmail.com is different then the SPF- and DKIM-validated domain of amazonses.com, DMARC authentication fails.

Closing Remarks

The research into DMARC, SPF, and DKIM took me on a rabbit hole and several days of headaches, pacing around the room, and talking to myself. But I am glad to uncover some of their secrets and understand a bit about how they operate at a lower level. After learning a bit more about the processes behind DMARC/SPF/DKIM, it now feels like a “Doh!” moment and all of the complexities just… vanished. It definitely took time and effort to understand, but now it’s amazing how some of the things that once seemed confusing are much clearer.

Think about the time you interacted with the Linux terminal for the first time, or tried to set up an Active Directory homelab without knowing what a domain controller was. I’d bet it was hella confusing at first. Each tiny step probably felt like punching through a concrete wall. But the thing about concrete walls is that they eventually crumble under persistent effort. While the process may be messy, leaving your hands figuratively dirty and bloody, the reward is a stronger body and a more resilient mind.

References

Unlike most blogs that discuss about DMARC, these are actually pretty good:

It’s also always interesting to read the RFCs as well! I’d highly recommend it since they are the original “source” of the technologies:

Taylor Nguyen

Taylor Nguyen

Cybersecurity and other stuff