The first electronic mail was sent in 1971 via the ARPANET. Since that time, progress has leaped forward and emails have become an inseparable part of daily life. By the end of 2019, the number of email users worldwide is expected to reach almost 3 billion people! Developers, in turn, set up things to let users send and receive emails. It's not rocket science, but you must know your oats with how email works. That's why the Mailtrap team decided to open the hood and guide you through the email basics that every developer must know.
Disclaimer: it would have been a very long text (much longer than it is) if I’d gone into detail for each subtopic. I decided to make it as brief as possible.
Email is a way to send some digital data from a sender to a recipient via the Internet. Both the sender and the recipient have a unique email address that corresponds to a standard format:
. For example,
An email provider is a service where you can open an email account and get an email address to send and receive electronic mail. Gmail is truly the most famous web-based email provider.
This is a very basic introduction to what email is. Now, let’s explore the more sophisticated things such as types of emails, what they consist of, and more.
Types of emails: marketing vs. transactional
If you ask Google how many types of emails there are, you’ll find versatile selections like five, seven, fifteen, and so on. From the developer’s perspective, marketing and transactional emails are in focus.
Marketing emails are not exactly what engineers regularly deal with. However, they form bulk mail, when the same content is delivered to a large number of recipients at the same time. Marketing emails include messages aimed at selling or promoting something. They include newsletters, product launch or feature updates, and so on.
Transactional emails are usually what developers have direct control over. They are generated and sent when a ‘transaction’ happens. Transactions include form submission, making a purchase, requesting a password reset, and more. The main value of transactional emails is that users want to receive them.
An email header is a section of code that contains specific data about the email’s origin. The recipient’s email client usually displays the partial or standard headers:
displays the sender’s email address.
displays the subject line of the email.
displays when (date and time) the email was created.
displays the recipient. In some cases, the recipient’s email address may be hidden.
(or Return-Path:) displays the email address for return mail.
stands for carbon copy. It is a standard header aimed at notifying the recipients about the content of the message.
stands for blind carbon copy. It specifies an additional copy to the recipient who will get the email and see other direct or copied recipients. The BCC: recipient, however, won’t be visible to other recipients.
The full headers contain the tech information about the email. Here are the most common ones:
displays the mailbox of a recipient the email that was delivered to.
displays when (date and time) the email was received by the email client.
displays a list of all servers the email was routed through. As a rule, the Received: header is represented by several lines, of which the last one is where the email originated and the first one is the email destination.
displays details on SPF authentication. The result can be
or pass for a received email, and
for a rejected one.
specifies information about the sender, the email, and the public key location required for authentication.
displays details on SMTP authentication. We’ll touch on this topic, as well as SPF and DKIM in the Email authentication explained section.
displays a unique string assigned to the email when created.
displays the email format like HTML, plaintext or multipart.
displays the actual content of the email.
displays information about any attachments.
displays the encoding scheme for the content.
-[Level/Score/Report]: displays information on how spammy the email is according to SpamAssassin.
Sample Email Headers
Delivered-To: [email protected] Received: by 2002:a02:c727:0:0:0:0:0 with SMTP id h7csp1944549jao; Wed, 23 Oct 2019 23:32:36 -0700 (PDT) Return-Path: <[email protected]> Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [220.127.116.11]) by mx.google.com with SMTPS id y1sor11976446oti.117.2019.10.23.23.32.35 for <[email protected]> (Google Transport Security); Wed, 23 Oct 2019 23:32:36 -0700 (PDT) Received-SPF: pass (google.com: domain of [email protected] designates 18.104.22.168 as permitted sender) client-ip=22.214.171.124; Authentication-Results: mx.google.com; dkim=pass [email protected] header.s=20161025 header.b="UnP/Lig1"; spf=pass (google.com: domain of [email protected] designates 126.96.36.199 as permitted sender) [email protected]; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=hrJ9yUbW0UidsTSyBmObH/qjSPBcHeZvop1/UXKVlUM=; b=UnP/Lig15hfjuV+5WGuJ1vfGscNuidgOpz+wP61KBaK3TK1bXbh4A1mX9LR028tD4S Pc5eP23vXRiwUnoXZ5qJI6ZVXzNejZbIprnZ42j6XvxqIo5nRd4OFqyaYt0unrFDEuvM MIME-Version: 1.0 From: The Sender <[email protected]> Date: Thu, 24 Oct 2019 09:32:54 +0300 Message-ID: <[email protected]om> Subject: Test email To: The Recipient <[email protected]> Cc: The RecipientCC <[email protected]> Content-Type: text/html; charset="US-ASCII"; name="Test email Headers.html" Content-Disposition: attachment; filename="Test email Headers.html" Content-Transfer-Encoding: base64 Content-ID: <f_k24bxtnk0> X-Attachment-Id: f_k24bxtnk0
Email content: plain text, HTML, images, encoding
header tells the email client what kind of data the email contains. The simplest option is the
. But the MIME protocol supports other types of content according to IANA:
header will indicate
if you use HTML in your email. HTML web development and HTML email building are different things. When you need to create HTML content for your email, you should bear in mind the list of specific rules:
For more on coding HTML emails, read Mailtrap’s Guide on How to Build HTML Email.
Multipart emails contain different types of content in one go. So, they have several
headers. Check out the following example:
MIME-Version: 1.0 From: The Sender <[email protected]> Date: Thu, 24 Oct 2019 09:32:54 +0300 Message-ID: <[email protected]om> Subject: Test email To: The Recipient <[email protected]> Content-Type: multipart/mixed; boundary="000000000000f372640595a23214" --000000000000f372640595a23214 Content-Type: multipart/alternative; boundary="000000000000f372620595a23212" --000000000000f372620595a23212 Content-Type: text/plain; charset="UTF-8" Hello --000000000000f372620595a23212 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <h2>Hello!<h2> --000000000000f372620595a23212-- --000000000000f372640595a23214 Content-Type: image/png; name="Capture001.png" Content-Disposition: attachment; filename="Capture001.png" Content-Transfer-Encoding: base64 Content-ID: <f_k24o133a0>
Here you can see five different
denotes that there is a mix of content. In our case, this is the plain text and HTML.
denotes that the email contains an attachment as well.
specifies the plain text part of the content.
specifies the HTML part of the content.
specifies the attachment part of the content.
Boundaries among the email content parts are defined with the
parameter, which is specified after
Attached or embedded images in emails
Enclosing and embedding are two basic approaches for attaching images to emails.
When you enclose an image, you add it as a separate file. The HTML structure is not affected, and the attached image remains unchanged. The drawback is that your email becomes heavier.
When you embed an image, it becomes an integral part of the HTML body. In this case, you do not have to worry about the email size, but your concerns shift to hosting the attachment and how it will be encoded.
Now, a couple of words about
. This header may have one of the following values:
- this is a default value, which implies no encoding. It ranges from 0000000 to 1111111, or 0 to 127 in base 10. That means ASCII 7-bit encoding has room for 128 different characters that can be represented as numbers. If the Content-Transfer-Encoding header is not present, then the 7bit encoding type is assumed.
- a 65-character subset of US-ASCII is used for encoding, enabling 6 bits to be represented per printable character. The base64 mechanism encodes data of any kind, and the output is represented in lines of no more than 76 characters each and is not human-readable at all.
- the mechanism encodes the data in such a way that the resulting octets are unlikely to be modified by mail transport. The encoded form of the data remains largely recognizable by humans.
In RFC 1341, there are also defined
values. They, however, have never been officially implemented. “There are no circumstances in which the
Content-Transfer-Encoding is actually legal on the Internet”. Besides, these mechanisms imply NO encoding.
Email template explained
An email template is a reusable HTML file for email campaigns. It allows you to tune up your email by adding styling elements such as images, cascading style sheets (CSS), and others. The main advantage of using email templates is time efficiency. With them, you produce your email campaigns faster, since most of the code is already there.
Maximum email size
When building an email, be it a transactional or a marketing one, you consider maximum email size limits. The first limit is for sending - it is set by your email service provider. The second limit is for retrieving - it is set by the email provider of your recipient. Limits are established mostly based on performance and security reasons. For example, with Gmail, you can send and retrieve emails up to 25MB. That’s why you may be willing to learn more about the ways to optimize email size limits.
In brief, each email comes through three checkpoints: the sender => the mail server => the recipient. If we expand this flow, we get the following:
All these transferring parties connect to each other via different email protocols.
Email protocols explained
There are three email protocols: SMTP, IMAP, and POP3. Of them, one is for sending, and two are for receiving. Simple Mail Transfer Protocol (SMTP) is meant only to send emails. With SMTP, an email gets from the sender to the email server and between servers. Once the email is already at the mail server, Internet Message Access Protocol (IMAP) or (Post Office Protocol version 3) POP3 comes into play to retrieve it from there.
Underneath each email delivery lies an SMTP conversation that consists of three stages:
The SMTP conversation is carried out using the commands from the client and responses from the server.
SMTP commands and responses
The client talks using commands that consist of alphabetical characters - for example,
, and others. The server responds with numerical codes such as
HELO, MAIL FROM
, and so on. The numerical code is followed by a text meant for a human user to get the point. For more on SMTP commands and replies, refer to RFC 5321.
220, 250, 354
Ports are communication endpoints commonly used to identify the exact location of an Internet address. Each email is sent to a dedicated server and a predefined SMTP port. There are four standard SMTP ports: 25, 465, 587, and 2525.
An SMTP server is a computer program or app that sends or relays outgoing emails using SMTP. There are cloud-based and self-hosted servers. Cloud-based is the most common option for numerous businesses and startups. Gmail, Mailgun, SendGrid, and many more are SMTP server providers.
For a self-hosted option, you need to build the whole email infrastructure of the SMTP server yourself. Usually, this is what large corporations opt for. At the same time, you can set up your own local SMTP server and use the email relay services for sending emails.
From a technical standpoint, SMTP relay is the process of transferring emails from one mail transfer agent to another. Here is a visual explanation of when SMTP relay comes into play.
In the context of email marketing, however, SMTP relay services usually refer to the actual SMTP servers.
An email queue is a system within the SMTP server that creates an array of emails waiting to be processed for delivery. Emails are put in a queue that provides temporary storage before processing. SMTP email queues decouple the sender from the recipient - they communicate without being connected.
SMTP relay vs. HTTP API
SMTP relay is the most common email transfer method for sending transactional and bulk emails. It is based on the Simple Mail Transfer Protocol and is a good solution for casual email senders who need simplicity for basic tasks.
Web API or HTTP API is a type of interface where the communication takes place using the Hypertext Transfer Protocol (HTTP). It makes email sending quicker because there is less back and forth required compared to the SMTP.
Email authentication explained
SMTP lacks a feature that would identify the origin of an email message and validate its domain. Hence, it can’t protect your app from frauds such as email spoofing, phishing, and spam. That’s why you need email authentication. SPF, DKIM, and DMARC are three widely adopted standards to authenticate emails.
SPF (Sender Policy Framework)
SPF is a protocol according to which the mail servers decide whether to receive or reject an incoming email. The information about authorized IP addresses within a particular domain is stored in the TXT record known as the SPF record. Here is a simple example of it:
v=spf1 +a +mx redirect=example.com -all
DKIM (DomainKeys Identified Mail)
DKIM is a digital signature that’s added to every email sent from a given email address. It’s a seemingly random set of characters that are hidden in the source code of an email. Here is how it may look:
DKIM-Signature: v=1; a=rsa-sha256; d=example.net; s=newyork; c=relaxed/simple; q=dns/txt; t=1117574938; x=1118006938; h=from:to:subject:date:keywords:keywords; bh=MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=; b=dzdVyOfAKCdLXdJOc9G2q8LoXSlEniSbav+yuU4zGeeruD00lszZ VoG4ZHRNiYzR
DMARC (Domain-Based Message Authentication Reporting and Conformance)
DMARC is not a self-consistent authentication method. It leverages DKIM and SPF to perform a more advanced check on each email received. Domain owners can specify their own authentication procedure known as a DMARC policy and instruct an incoming server on what to do if an email fails to pass the DMARC test. Here is an example of a DMARC record:
v=DMARC1; p=reject; rua=mailto:[email protected],mailto:[email protected],mailto:[email protected]; ruf=mailto:[email protected]
SMTP authentication or SMTP AUTH
SMTP AUTH is the service extension of the ESMTP. It requires that an email sender (client) must have permission to use the email server. Authentication is carried out according to the SASL mechanism. The most common ones are PLAIN, LOGIN, and CRAM-MD5, but there are also many others.
SMTP comes with no security features. This entails vulnerability to different forms of threats and attacks. For example, malevolent people can access your emails or use your SMTP server for sending spam. Email providers are interested in protecting their customers and use security layers such as SSL or TLS. Another approach to securing email communication is to use an end-to-end encryption method such as S/MIME, PGP, and others.
Email encryption methods
Secure Socket Layer or SSL is a cryptographic protocol used in email transmission. It turns emails into unreadable strings of characters, so no one could use them if intercepted.
Transport Layer Security or TLS is a successor of SSL. It offers a more reliable security profile.
Unlike those above, STARTTLS is not an encryption protocol but an SMTP command that tells an SMTP server that an email client wants to use a secure connection using SSL or TLS.
Read about SSL, TLS, and STARTTLS to learn more about encryption in email transmissions.
If you’re planning to send your email to thousands of recipients, testing is the first thing that needs to be done. For this, you can follow a simple email testing checklist, consisting of:
Testing the email sending capability
In this part, you need to test two things: whether your app is capable of sending emails, and whether the mail server functions well.
Sending test emails
A proper SMTP server for testing is a fake one. It can make your email testing safe and easy. But the main advantage is that your test emails won’t go outside your staging environment. There are two ways to set up a fake SMTP server for testing:
As an alternative, you can use a real SMTP server combined with temporary email accounts known as dummy emails. Services such as Mailinator or Maildrop can help you create those. Whereas this approach may be good for testing a very simple web app, you shouldn’t go with it for more complex projects. Why? There are many reasons such as a necessity to create hundreds of test email accounts, a short lifetime of those, plain text support only, and others.
SMTP server testing
Let’s say your app has successfully passed the test for email sending. Now, you need to check whether the SMTP server responds to the client commands and can send emails. This can be done manually using Telnet or you can opt for a dedicated online service such as MXToolbox or Wormly.
Also, do not neglect to test your SMTP server for relaying. This includes checking the SMTP connection and the authentication mechanism. For this, you can also choose either an installable or web-based tool.
Deliverability or spam testing
Delivery denotes whether an email was accepted by a recipient server. If your email bounced soft or hard, it was not necessarily your fault. In some cases, the failed delivery can be out of the sender’s control.
Deliverability denotes whether an email ended up in the inbox instead of a spam folder. So, you need to test how spammy your email is. Tools such as Spam Check or SendForensics allow you to check this and point out some weak points. At the same time, you may need to do some work and implement a few of the best practices to improve email deliverability.
Test email content
The final stage of email testing focuses on checking text and formatting. What you can do here is:
Something else you may be interested in is Bcc testing. The emails meant for Bcc’ed recipients may get to the spam folder, and there was no functionality to test and debug this. The Mailtrap team, however, marked this issue and delivered a feature to test Bcc in emails.
This email basics tutorial is designed for developers. However, it will be of use to anyone looking for layman’s terms of many email-related things in one place. Good luck with your email sending stuff!
P.S. I will highly appreciate your feedback so don't hesitate to share it with me on Linkedin :)