Solving the problem of vulnerabilities & compliance when using Open Source in product development
Sending mail might sound easy, but to avoid getting your mail caught in spam filters, and to prevent others from sending spoofed email in your name, you need to employ different preventive methods. One such method is DMARC, which allows domain administrators to apply policies with regard to email authentication. You also have the possibility to get reports sent to you with the results of the applied policy.
This article shows how you can use existing open-source tools to visualize these reports in a graphical way, self-hosted on your own servers, without having to submit your reports to a third party.
The email protocols used today were constructed in a time where computers and users on the network were considered trusted. Today, this causes problems with unsolicited and spoofed email, which pose a threat to users all over the world.
Several methods have been proposed to provide email authentication. DMARC (Domain-based Message Authentication, Reporting and Conformance) is one such protocol. DMARC allows a domain owner to publish a policy of requirements that email sent from this domain should fullfil. The receiving mail server will then evaluate the policy, and if the policy does not match, this could indicate that the email was spoofed. The receiving mail server may then take other actions, such as marking it as spam or discarding it.
DMARC is based on the use of two other methods, namely SPF (Sender Policy Framework) and DKIM (DomainKeys Identified Mail). DMARC ties them together by allowing a domain administrator to state if any or both are expected to be used for sending email with a particular domain as sender.
Briefly, SPF is used to set a predefined set of mail servers as authorized to send email for a particular domain. The SPF record is set on the domain, and the receiving mail server can then check if the email originated from the expected mail server by matching the IP address or the domain name of the server.
DKIM instead adds a signature to the email at the sender mail server. This signature can then be verified by the receiving server, to see if it matches. This also gives an assurance that the email originated from the expected domain, since only mail servers on that domain should be able to create signatures.
The DMARC policy supports the sending of DMARC reports, where the receiving mail server sends a (usually) daily report of mail received from a particular domain. These reports can be sent by mail, and if configured as in the example record below, email@example.com would receive these reports.
_dmarc.example.com. 7200 IN TXT "v=DMARC1; p=none; rua=mailto:firstname.lastname@example.org; pct=100"
The DMARC reports are sent as XML files, but are not very practical for humans to read, especially not since you may receive several of them every day. Instead, what we want to do is to visualize this in a more human-friendly way.
After searching the internet for projects that parse DMARC reports, I started looking at parsedmarc, an open-source project hosted on Github. It has a lot of desirable features, for example:
While templates are provided inside the Git repository, connecting the different components are non-trivial. You would have to run parsedmarc, set up an Elasticsearch instance, and then connect this to a Grafana instance to visualize it. This requires a lot of manual configuration if you just want to try things out.
However, I will now show how you can connect these components together using Docker.
To simplify things a bit, I decided to use docker-compose to connect three different components together:
In this way, you can simply spin up a series of Docker containers, without a manual installation process, which provides a Grafana dashboard available in the browser.
I have prepared a repository with the required files available on Github at the following address https://github.com/debricked/dmarc-visualizer. The docker-compose.yml file looks like this:
version: '3.7' services: parsedmarc: build: ./parsedmarc/ volumes: - ./files:/input:ro - ./output_files:/output command: parsedmarc -c /parsedmarc.ini /input/* depends_on: - elasticsearch elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0 environment: - discovery.type=single-node grafana: build: ./grafana/ ports: - 3000:3000 environment: GF_INSTALL_PLUGINS: grafana-piechart-panel,grafana-worldmap-panel GF_AUTH_ANONYMOUS_ENABLED: 'true'
In this example, I have chosen to read all DMARC reports from a directory called
, which will be parsed by
. If desired, you could also configure parsedmarc to read directly from your inbox.
The aggregated results are stored in Elasticsearch. After this, Grafana is configured with the dashboard from parsedmarc, and also configured to connect to the Elasticsearch instance.
This configuration is done in
directory, if you want to see the details.
(An additional note, if you want to be able to see geographical information from the DMARC reports, i.e., the country corresponding to certain IP-addresses, you need to register and download the GeoIP2 database yourself, since we cannot redistribute it. You can find instructions here https://dev.maxmind.com/geoip/geoip2/geolite2/.
You can then copy the correct files to
You can now bring up the containers with
. It will take a while for all containers to start, and you might temporarily see some errors until Elasticsearch is up, but after this, parsedmarc will start to parse all DMARC reports. When the parsedmarc container has exited successfully, you can visit http://localhost:3000 to see the Grafana dashboard.
Click Home in the top-left corner, and then click on the “DMARC Reports” dashboard. You will see the following view:
As you can see, you can view a lot of information, for example the individual SPF and DKIM passage, as well as the final DMARC passage. Recall that DMARC Passage can be true if either SPF or DKIM alignment is true. You can also see graphs over time, so that you can monitor how alignment changes over time, for example after a policy or configuration change.
In this article I have showed how you can combine some great open-source resources together using Docker, to get a solution that is easy to deploy on your own system. Implementing all parts yourself would have been a huge task, but now when I could use these already existing components, I could quite quickly analyse our ever growing pile of DMARC reports.
Interested in more how to’s? Visit our blog!
Previously published at https://debricked.com/blog/2020/05/14/analyse-and-visualize-dmarc-results-using-open-source-tools/