This page contains a technical reference for how DMARCwise processes DMARC reports, especially when it comes to normalization and correction of invalid reports. It may in some cases help explain discrepancies you may see in the dashboard compared to the original reports.
Once received as an attachment to an email message, a DMARC report is parsed and passed through a validator.
The validator is typically lax and forgives many issues that would be unacceptable according to the DMARC specification. Some fields in the XML reports are however validated more strictly, otherwise the report would be very hard to use in practice. This includes for example the policy fields, which must contain a valid policy name.
Although blatantly invalid reports are rare, when one such report is received it is not imported and is retained for manual review.
Corrections
To prevent reports from being discarded for minor reasons, we apply corrections, which fall into three categories:
- Normalization: a field is cleaned up with the aim of improving compatibility or making the data more usable. Examples include lowercasing values, removing invalid punctuation, accepting typos. These are high-confidence harmless corrections.
- Recovered: a field contains an invalid value that violates the DMARC specification, but the intent is clear enough that we can replace it with a valid value. Our judgement is that replacing or inferring the intended value is more useful than discarding the report.
- Discarded: an individual unusable or invalid item is removed so the rest of the report can still be processed. An example is discarding a DKIM auth result which contains no data. These items are usually the result of bugs in the software that generated the report.
These corrections are based on bad reports we’ve seen in the past and are updated when new invalid reports come through.
When you browse a report in DMARCwise, you already see the normalized/corrected data. When a correction was applied, a badge is shown and clicking on it lets you see how the data was modified.
You can also always see the original unprocessed report in XML format.
Our implementation of the parsing, correction and validation of DMARC reports is thoroughly tested with automated tests.
List of corrections
The tables below list the corrections we currently apply to new reports we receive, grouped by category. Each row references the original DMARC XML element (using the path from the report’s <feedback> root) and explains what happens to it. Note that these rules change over time, so you may see different corrections in older reports.
Normalization
| XML element | Correction |
|---|---|
policy_published/p | Surrounding semicolons are trimmed, e.g. reject; becomes reject. |
policy_published/p | An empty value is treated as missing (as if the p tag was absent). |
policy_published/sp | Surrounding semicolons are trimmed. |
policy_published/sp | An empty value is treated as missing. |
policy_published/pct | A trailing percent sign is trimmed, e.g. 100% becomes 100. |
policy_published/pct | An empty value is treated as missing. |
policy_published/aspf | An empty value is treated as missing. |
policy_published/adkim | An empty value is treated as missing. |
record/identities | The misspelled identities element is renamed to the correct identifiers. |
record/auth_results/spf/result | Lowercased. |
record/auth_results/dkim/domain | Lowercased. |
record/auth_results/dkim/selector | Lowercased. |
record/auth_results/dkim/result | Lowercased. |
Recovered
| XML element | Correction |
|---|---|
policy_published/p | The invalid value unknown is treated as missing (as if the p tag was absent). |
policy_published/sp | The invalid value unknown is treated as missing. |
policy_published/aspf | The invalid value unknown is treated as missing. |
policy_published/adkim | The invalid value unknown is treated as missing. |
record/identifiers/header_from | When empty or missing, it is filled with the value of envelope_from. |
record/row/policy_evaluated/dkim | A non-pass DKIM result (none, neutral, temperror, permerror) is replaced with fail. The alignment result can only be pass or fail, and alignment cannot exist without a DKIM pass, so any recognized non-pass result is treated as an alignment failure. |
record/row/policy_evaluated/spf | A non-pass SPF result (none, neutral, softfail, temperror, permerror) is replaced with fail. The alignment result can only be pass or fail, and alignment cannot exist without an SPF pass, so any recognized non-pass result is treated as an alignment failure. |
Discarded
| XML element | Correction |
|---|---|
record | When the report contains a single record row whose count is 0, the record is discarded but the report is kept. These records are typically completely empty and would fail validation. |
record/auth_results/spf | Discarded when its result is empty. |
record/auth_results/spf | Discarded when its domain is empty. |
record/auth_results/dkim | Discarded when its result is none: a result of none means the message was not signed, so it makes little sense to present it as a DKIM signature. |
record/auth_results/dkim | Discarded when its result is empty. |
record/auth_results/dkim | Discarded when its domain is empty. |