The A-Z Of Email Development And Sending Emails

The first electronic mail was sent in 1971 via the ARPANET. Since that time, progress has leaped forward and emails have become an inseparable part of daily life. By the end of 2019, the number of email users worldwide is expected to reach almost 3 billion people! Developers, in turn, set up things to let users send and receive emails. It's not rocket science, but you must know your oats with how email works. That's why the Mailtrap team decided to open the hood and guide you through the email basics that every developer must know.

Disclaimer: it would have been a very long text (much longer than it is) if I’d gone into detail for each subtopic. I decided to make it as brief as possible.

Email basics

Email is a way to send some digital data from a sender to a recipient via the Internet. Both the sender and the recipient have a unique email address that corresponds to a standard format:

<user-name>@<email-provider-domain>

. For example,

[email protected]

An email provider is a service where you can open an email account and get an email address to send and receive electronic mail. Gmail is truly the most famous web-based email provider.

This is a very basic introduction to what email is. Now, let’s explore the more sophisticated things such as types of emails, what they consist of, and more.

Types of emails: marketing vs. transactional

If you ask Google how many types of emails there are, you’ll find versatile selections like five, seven, fifteen, and so on. From the developer’s perspective, marketing and transactional emails are in focus.

Marketing emails are not exactly what engineers regularly deal with. However, they form bulk mail, when the same content is delivered to a large number of recipients at the same time. Marketing emails include messages aimed at selling or promoting something. They include newsletters, product launch or feature updates, and so on.

Transactional emails are usually what developers have direct control over. They are generated and sent when a ‘transaction’ happens. Transactions include form submission, making a purchase, requesting a password reset, and more. The main value of transactional emails is that users want to receive them.

Email headers, Cc, and Bcc explained

An email header is a section of code that contains specific data about the email’s origin. The recipient’s email client usually displays the partial or standard headers:

```
From:
```
displays the sender’s email address.
```
Subject:
```
displays the subject line of the email.
```
Date:
```
displays when (date and time) the email was created.
```
To:
```
displays the recipient. In some cases, the recipient’s email address may be hidden.
```
Reply To:
```
(or Return-Path:) displays the email address for return mail.
```
CC: 
```
stands for carbon copy. It is a standard header aimed at notifying the recipients about the content of the message.
```
BCC: 
```
stands for blind carbon copy. It specifies an additional copy to the recipient who will get the email and see other direct or copied recipients. The BCC: recipient, however, won’t be visible to other recipients.

The full headers contain the tech information about the email. Here are the most common ones:

```
Envelope-To:
```
displays the mailbox of a recipient the email that was delivered to.
```
Delivery Date:
```
displays when (date and time) the email was received by the email client.
```
Received:
```
displays a list of all servers the email was routed through. As a rule, the Received: header is represented by several lines, of which the last one is where the email originated and the first one is the email destination.
```
Received-SPF:
```
displays details on SPF authentication. The result can be
```
neutral
```
or pass for a received email, and
```
fail
```
for a rejected one.
```
DKIM-Signature:
```
specifies information about the sender, the email, and the public key location required for authentication.
```
Authentication-Results: 
```
displays details on SMTP authentication. We’ll touch on this topic, as well as SPF and DKIM in the Email authentication explained section.
```
Message-ID:
```
displays a unique string assigned to the email when created.
MIME-Version: indicates that the rules of Multipurpose Internet Mail Extensions are followed. MIME is an Internet standard that extends the format of email. The value of this header is constantly 1.0.
```
Content-Type:
```
displays the email format like HTML, plaintext or multipart.
```
Message Body:
```
displays the actual content of the email.
```
Content-Disposition: 
```
displays information about any attachments.
```
Content-Transfer-Encoding:
```
displays the encoding scheme for the content.
```
X-Spam
```
-[Level/Score/Report]: displays information on how spammy the email is according to SpamAssassin.

Sample Email Headers

Delivered-To: [email protected]
Received: by 2002:a02:c727:0:0:0:0:0 with SMTP id h7csp1944549jao;
        Wed, 23 Oct 2019 23:32:36 -0700 (PDT)
Return-Path: <[email protected]>
Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41])
        by mx.google.com with SMTPS id y1sor11976446oti.117.2019.10.23.23.32.35
        for <[email protected]>
        (Google Transport Security);
        Wed, 23 Oct 2019 23:32:36 -0700 (PDT)
Received-SPF: pass (google.com: domain of [email protected] designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41;
Authentication-Results: mx.google.com;
       dkim=pass [email protected] header.s=20161025 header.b="UnP/Lig1";
       spf=pass (google.com: domain of [email protected] designates 209.85.220.41 as permitted sender) [email protected];
       dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20161025;
        h=mime-version:from:date:message-id:subject:to:cc;
        bh=hrJ9yUbW0UidsTSyBmObH/qjSPBcHeZvop1/UXKVlUM=;
        b=UnP/Lig15hfjuV+5WGuJ1vfGscNuidgOpz+wP61KBaK3TK1bXbh4A1mX9LR028tD4S
         Pc5eP23vXRiwUnoXZ5qJI6ZVXzNejZbIprnZ42j6XvxqIo5nRd4OFqyaYt0unrFDEuvM
MIME-Version: 1.0
From: The Sender <[email protected]>
Date: Thu, 24 Oct 2019 09:32:54 +0300
Message-ID: <CAB4io_0Y3oRZUgd=Lb08LnQCaHa2vFvKmVsc1xV53N-Etb_Yrg@mail.gmail.com>
Subject: Test email
To: The Recipient <[email protected]>
Cc: The RecipientCC <[email protected]>
Content-Type: text/html; charset="US-ASCII"; name="Test email Headers.html"
Content-Disposition: attachment; filename="Test email Headers.html"
Content-Transfer-Encoding: base64
Content-ID: <f_k24bxtnk0>
X-Attachment-Id: f_k24bxtnk0

Email content: plain text, HTML, images, encoding

The

Content-Type

header tells the email client what kind of data the email contains. The simplest option is the

text/plain

. But the MIME protocol supports other types of content according to IANA:

application (for example,
```
application/json
```
)
audio (for example,
```
audio/mpeg
```
)
font (for example,
```
font/ttf
```
)
example (any subtype following the media type syntax may be used in examples)
image (for example,
```
image/png
```
)
message (for example,
```
message/global-delivery-status
```
)
model (for example,
```
model/gltf-binary
```
)
multipart (for example,
```
multipart/form-data
```
)
text (for example,
```
text/css
```
)
video (for example,
```
video/mp4
```
)

HTML email

The

Content-Type

header will indicate

text/html

if you use HTML in your email. HTML web development and HTML email building are different things. When you need to create HTML content for your email, you should bear in mind the list of specific rules:

No rendering standards are applied since recipients may use different email clients.
An HTML email will be rendered only once it ends up in the inbox. So, it must come through spam filters.
An HTML email should be responsive – it should look as designed on any device and on any screen size.
An HTML email should be previewed before sending.

For more on coding HTML emails, read Mailtrap’s Guide on How to Build HTML Email.

Multipart Email

Multipart emails contain different types of content in one go. So, they have several

Content-Type

headers. Check out the following example:

MIME-Version: 1.0
From: The Sender <[email protected]>
Date: Thu, 24 Oct 2019 09:32:54 +0300
Message-ID:
<CAB4io_0Y3oRZUgd=Lb08LnQCaHa2vFvKmVsc1xV53N-Etb_Yrg@mail.gmail.com>
Subject: Test email
To: The Recipient <[email protected]>
Content-Type: multipart/mixed; boundary="000000000000f372640595a23214"
--000000000000f372640595a23214
Content-Type: multipart/alternative; boundary="000000000000f372620595a23212"
--000000000000f372620595a23212
Content-Type: text/plain; charset="UTF-8"
Hello
--000000000000f372620595a23212
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<h2>Hello!<h2>
--000000000000f372620595a23212--
--000000000000f372640595a23214
Content-Type: image/png; name="Capture001.png"
Content-Disposition: attachment; filename="Capture001.png"
Content-Transfer-Encoding: base64
Content-ID: <f_k24o133a0>

Here you can see five different

Content-Type

headers:

```
multipart/mixed
```
denotes that there is a mix of content. In our case, this is the plain text and HTML.
```
multipart/alternative
```
denotes that the email contains an attachment as well.
```
text/plain
```
specifies the plain text part of the content.
```
text/html
```
specifies the HTML part of the content.
```
image/png
```
specifies the attachment part of the content.

Boundaries among the email content parts are defined with the

boundary=””

parameter, which is specified after

Content-Type multipart/.

Attached or embedded images in emails

Enclosing and embedding are two basic approaches for attaching images to emails.

When you enclose an image, you add it as a separate file. The HTML structure is not affected, and the attached image remains unchanged. The drawback is that your email becomes heavier.

When you embed an image, it becomes an integral part of the HTML body. In this case, you do not have to worry about the email size, but your concerns shift to hosting the attachment and how it will be encoded.

Content encoding

Now, a couple of words about

Content-Transfer-Encoding

. This header may have one of the following values:

```
7bit
```
- this is a default value, which implies no encoding. It ranges from 0000000 to 1111111, or 0 to 127 in base 10. That means ASCII 7-bit encoding has room for 128 different characters that can be represented as numbers. If the Content-Transfer-Encoding header is not present, then the 7bit encoding type is assumed.
```
base64
```
- a 65-character subset of US-ASCII is used for encoding, enabling 6 bits to be represented per printable character. The base64 mechanism encodes data of any kind, and the output is represented in lines of no more than 76 characters each and is not human-readable at all.
```
quoted-printable
```
- the mechanism encodes the data in such a way that the resulting octets are unlikely to be modified by mail transport. The encoded form of the data remains largely recognizable by humans.

In RFC 1341, there are also defined

8bit

and

binary

values. They, however, have never been officially implemented. “There are no circumstances in which the

8bit

binary

Content-Transfer-Encoding is actually legal on the Internet”. Besides, these mechanisms imply NO encoding.

Email template explained

An email template is a reusable HTML file for email campaigns. It allows you to tune up your email by adding styling elements such as images, cascading style sheets (CSS), and others. The main advantage of using email templates is time efficiency. With them, you produce your email campaigns faster, since most of the code is already there.

Maximum email size

When building an email, be it a transactional or a marketing one, you consider maximum email size limits. The first limit is for sending - it is set by your email service provider. The second limit is for retrieving - it is set by the email provider of your recipient. Limits are established mostly based on performance and security reasons. For example, with Gmail, you can send and retrieve emails up to 25MB. That’s why you may be willing to learn more about the ways to optimize email size limits.

Basics of sending and receiving email

In brief, each email comes through three checkpoints: the sender => the mail server => the recipient. If we expand this flow, we get the following:

A mail user agent (MUA) hands over the email to the mail submission agent (MSA).
From the MSA, the email gets to the mail transfer agent (MTA). Both the MTA and MSA are hosted on the same SMTP server.
The MTA may transfer the email to another MTA or right to the mail delivery agent (MDA).
The MDA delivers the email when the recipient’s MUA requests it.

All these transferring parties connect to each other via different email protocols.

Email protocols explained

There are three email protocols: SMTP, IMAP, and POP3. Of them, one is for sending, and two are for receiving. Simple Mail Transfer Protocol (SMTP) is meant only to send emails. With SMTP, an email gets from the sender to the email server and between servers. Once the email is already at the mail server, Internet Message Access Protocol (IMAP) or (Post Office Protocol version 3) POP3 comes into play to retrieve it from there.

SMTP basics

Underneath each email delivery lies an SMTP conversation that consists of three stages:

SMTP handshake - the SMTP client establishes a TCP connection with the SMTP server.
Email transfer.
Termination – the client and the server say goodbye to each other.

The SMTP conversation is carried out using the commands from the client and responses from the server.

SMTP commands and responses

The client talks using commands that consist of alphabetical characters - for example,

HELO, MAIL FROM

, and others. The server responds with numerical codes such as

220, 250, 354

, and so on. The numerical code is followed by a text meant for a human user to get the point. For more on SMTP commands and replies, refer to RFC 5321.

SMTP ports

Ports are communication endpoints commonly used to identify the exact location of an Internet address. Each email is sent to a dedicated server and a predefined SMTP port. There are four standard SMTP ports: 25, 465, 587, and 2525.

SMTP server

An SMTP server is a computer program or app that sends or relays outgoing emails using SMTP. There are cloud-based and self-hosted servers. Cloud-based is the most common option for numerous businesses and startups. Gmail, Mailgun, SendGrid, and many more are SMTP server providers.

For a self-hosted option, you need to build the whole email infrastructure of the SMTP server yourself. Usually, this is what large corporations opt for. At the same time, you can set up your own local SMTP server and use the email relay services for sending emails.

SMTP relay

From a technical standpoint, SMTP relay is the process of transferring emails from one mail transfer agent to another. Here is a visual explanation of when SMTP relay comes into play.

In the context of email marketing, however, SMTP relay services usually refer to the actual SMTP servers.

SMTP queues

An email queue is a system within the SMTP server that creates an array of emails waiting to be processed for delivery. Emails are put in a queue that provides temporary storage before processing. SMTP email queues decouple the sender from the recipient - they communicate without being connected.

SMTP relay vs. HTTP API

SMTP relay is the most common email transfer method for sending transactional and bulk emails. It is based on the Simple Mail Transfer Protocol and is a good solution for casual email senders who need simplicity for basic tasks.

Web API or HTTP API is a type of interface where the communication takes place using the Hypertext Transfer Protocol (HTTP). It makes email sending quicker because there is less back and forth required compared to the SMTP.

Email authentication explained

SMTP lacks a feature that would identify the origin of an email message and validate its domain. Hence, it can’t protect your app from frauds such as email spoofing, phishing, and spam. That’s why you need email authentication. SPF, DKIM, and DMARC are three widely adopted standards to authenticate emails.

SPF (Sender Policy Framework)

SPF is a protocol according to which the mail servers decide whether to receive or reject an incoming email. The information about authorized IP addresses within a particular domain is stored in the TXT record known as the SPF record. Here is a simple example of it:

v=spf1 +a +mx redirect=example.com -all

DKIM (DomainKeys Identified Mail)

DKIM is a digital signature that’s added to every email sent from a given email address. It’s a seemingly random set of characters that are hidden in the source code of an email. Here is how it may look:

DKIM-Signature: v=1; a=rsa-sha256; d=example.net; s=newyork;
     c=relaxed/simple; q=dns/txt; t=1117574938; x=1118006938;
     h=from:to:subject:date:keywords:keywords;
     bh=MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=;
    b=dzdVyOfAKCdLXdJOc9G2q8LoXSlEniSbav+yuU4zGeeruD00lszZ
              VoG4ZHRNiYzR

DMARC (Domain-Based Message Authentication Reporting and Conformance)

DMARC is not a self-consistent authentication method. It leverages DKIM and SPF to perform a more advanced check on each email received. Domain owners can specify their own authentication procedure known as a DMARC policy and instruct an incoming server on what to do if an email fails to pass the DMARC test. Here is an example of a DMARC record:

v=DMARC1; p=reject; rua=mailto:[email protected],mailto:[email protected],mailto:[email protected]; ruf=mailto:[email protected]

SMTP authentication or SMTP AUTH

SMTP AUTH is the service extension of the ESMTP. It requires that an email sender (client) must have permission to use the email server. Authentication is carried out according to the SASL mechanism. The most common ones are PLAIN, LOGIN, and CRAM-MD5, but there are also many others.

Email security explained

SMTP comes with no security features. This entails vulnerability to different forms of threats and attacks. For example, malevolent people can access your emails or use your SMTP server for sending spam. Email providers are interested in protecting their customers and use security layers such as SSL or TLS. Another approach to securing email communication is to use an end-to-end encryption method such as S/MIME, PGP, and others.

Email encryption methods

Secure Socket Layer or SSL is a cryptographic protocol used in email transmission. It turns emails into unreadable strings of characters, so no one could use them if intercepted.

Transport Layer Security or TLS is a successor of SSL. It offers a more reliable security profile.

Unlike those above, STARTTLS is not an encryption protocol but an SMTP command that tells an SMTP server that an email client wants to use a secure connection using SSL or TLS.

Read about SSL, TLS, and STARTTLS to learn more about encryption in email transmissions.

Email testing checklist

If you’re planning to send your email to thousands of recipients, testing is the first thing that needs to be done. For this, you can follow a simple email testing checklist, consisting of:

Testing the email sending capability
SMTP server testing
Test emails sending
Spam testing
Email content testing

Testing the email sending capability

In this part, you need to test two things: whether your app is capable of sending emails, and whether the mail server functions well.

Sending test emails

A proper SMTP server for testing is a fake one. It can make your email testing safe and easy. But the main advantage is that your test emails won’t go outside your staging environment. There are two ways to set up a fake SMTP server for testing:

The first one is to adopt a ready-to-use cloud-based solution or SaaS. You may consider a professional service such as Mailtrap.io. It has a dummy SMTP server underneath and provides a set of other useful features like spam analysis, HTML check, and many more.
The second way is to install a local SMTP server for testing. The popular solutions include MailCatcher and MailHog. They’re flexible in use, but you’ll have to put in some time and effort setting them up.

As an alternative, you can use a real SMTP server combined with temporary email accounts known as dummy emails. Services such as Mailinator or Maildrop can help you create those. Whereas this approach may be good for testing a very simple web app, you shouldn’t go with it for more complex projects. Why? There are many reasons such as a necessity to create hundreds of test email accounts, a short lifetime of those, plain text support only, and others.

SMTP server testing

Let’s say your app has successfully passed the test for email sending. Now, you need to check whether the SMTP server responds to the client commands and can send emails. This can be done manually using Telnet or you can opt for a dedicated online service such as MXToolbox or Wormly.

Also, do not neglect to test your SMTP server for relaying. This includes checking the SMTP connection and the authentication mechanism. For this, you can also choose either an installable or web-based tool.

Deliverability or spam testing

Delivery denotes whether an email was accepted by a recipient server. If your email bounced soft or hard, it was not necessarily your fault. In some cases, the failed delivery can be out of the sender’s control.

Deliverability denotes whether an email ended up in the inbox instead of a spam folder. So, you need to test how spammy your email is. Tools such as Spam Check or SendForensics allow you to check this and point out some weak points. At the same time, you may need to do some work and implement a few of the best practices to improve email deliverability.

Test email content

The final stage of email testing focuses on checking text and formatting. What you can do here is:

Preview you email to learn how it will look on different devices, browsers, or email clients.
Validate the markup (HTML, XHTML, and CSS) of your email.
Brush up the body copy and translation (yes, spam filters check the grammar, as well).
Test the subject line.
Split test your email campaign.

Something else you may be interested in is Bcc testing. The emails meant for Bcc’ed recipients may get to the spam folder, and there was no functionality to test and debug this. The Mailtrap team, however, marked this issue and delivered a feature to test Bcc in emails.

To wrap up

This email basics tutorial is designed for developers. However, it will be of use to anyone looking for layman’s terms of many email-related things in one place. Good luck with your email sending stuff!

P.S. I will highly appreciate your feedback so don't hesitate to share it with me on Linkedin :)