The Ultimate Guide to Webhooks

Goals

By the end of this guide, you should:

✅ Gain a better understanding of webhooks — what they are and when to use them
✅ Gain a better understanding of the security measures for using webhooks in practice
✅ Gain a better understanding of HS256

Introduction

Most modern applications we build today require integration with at least one or more 3rd party service(s).

These may include services such as:

Stripe (Payments)
SendGrid (Email)
Auth0 (Accounts)
HubSpot or Salesforce - Customer relation management (CRM)
So forth…

Because of this, our application is actually designed in a very modular way — or in other words, using microservices.

Now comes the question of how we coordinate between these services. Even more importantly, how do we keep the data in sync?

Well, there are many ways to achieve this. A very common technique, recommended by many 3rd party services, is using a webhook.

This technique allows us to keep the data in both systems in alignment over time as they change by using an HTTP request.

This is what we will explore in this guide. Let’s take a look at how it works.

The Webhook

The webhook itself is nothing special, it is leveraging the HTTP protocol in order to transfer the data over the network to keep the data in alignment.

It uses what is referred to as a “push model”, where the updates from the 3rd party service is pushed to the receiving endpoint (your application).

The HTTP request consists of:

URL - Where to send the data
Body - The data sent
HTTP header - Details about how the data is formatted
Request Method - The HTTP method, it is typically using a POST method for webhooks

To get a better idea of this technique, let’s go through an example using Stripe.

Example - Stripe webhook

1. The Registration

When we first begin to integrate a webhook with Stripe, we need to register our URL.

This is the API endpoint that Stripe will use to push their updates to our application.

2. Pushing the event

Once we’ve registered the webhook, we can start receiving updates.

First, we would make a payment request to Stripe to initialize a payment transaction.

Then, on Stripe’s service, they would receive that request and then process it whenever their system gets around to it.

Once the transaction has been processed, it would push the updates to our webhook URL.

This would include information such as the status of that payment transaction, and other relevant details about the transaction.

3. Receiving the event

Once we’ve received the update on our end, we can do whatever we like with the latest data.

That’s really what webhooks do in a nutshell.

Now, in practice, there are a few things to consider such as security.

Let’s take a look at what are some of the common attack vectors and the approaches to take in order to mitigate them.

Security

Here are a few security attack vectors we should be mindful of when it comes to creating a webhook.

The attack vectors are:

Person-in-the-middle attacks
Unauthorized Access
Replay attacks

Person-in-the-middle attacks

To ensure our connection is safe from person-in-the-middle attacks where malicious users can sniff the traffic and gain details about the request.

How do we overcome this?

You should always ensure that the communication channel between your application and the 3rd party service is encrypted. This means using HTTPS (TLS/SSL).

Unauthorized Access

Using the webhook to receive updates likely means that we will need to expose a public endpoint.

So, how do we ensure that the events that we receive are only from a trusted and valid source? This is where authentication comes in.

There are a few methods that can be used to achieve this.

API token

Depending on the 3rd party service, some may provide an option to register a webhook with an API token or even generate one for you.

This token will be added to the Authorization header, so that you can verify the webhook requests that you receive.

This technique is not very common among the various 3rd party services that offer webhooks. Nevertheless, it is still an option some 3rd party services offer.

Digital signatures - HS256

Another technique for authentication is using HS256.

This technique is more common across the various 3rd party services, where it makes use of digital signatures for verification.

So, rather than using a token, it makes use of cryptographic signatures (or hashes, checksums, digital signatures) to ensure that the request is coming from a valid source by verifying the signature from the sender.

HS256 stands for HMAC with SHA-256.

HMAC stands for hash-based message authentication code, which allows parties or systems to establish a trusting relationship (If you like to learn more, check out the in-depth explanation below).

HMAC requires the following:

A shared secret (or key)
A Hashing algorithm (in our case it would be SHA-256)

Signature formula: SHA256(data, secret)

A good analogy of this process would be like a coat check at a venue, where the attendant would hand you a piece of paper with a number.

When you are ready to get your coat, you’d hand the attendant the number you have and you can retrieve your coat.

So, in this process, instead of getting verified through your identity, you were verified by the piece of paper with the number.

Let’s revisit our example with Stripe again but now with signature verification.

A webhook process with signature Verification:

Register with a webhook with Stripe with a URL and shared secret
Make a payment request to Stripe from our application
Stripe would process it, then proceed to generate a signature by hashing the payload and the shared secret with SHA-256
Stripe then sends the webhook request, and adds the signature to the header (ie Stripe-Signature )
Our application receives the event, and we re-generate the signature using the payload and the shared secret with SHA-256
Now we match the generated signature against the signature received through the header of the request
If it matches then we are good to go, otherwise, we should reject the request

IP restrictions

Another form of restricting access to our endpoint is adding an allowed list of trusted IPs. This gives us more control over which clients can make requests to the endpoints.

That said, this doesn’t replace having a method for authentication; however, it does reduce the attack surface.

Replay attacks

You can probably guess what the security concern is just from the name.

Basically, it is an attack where malicious users somehow get access to a request that has been already received and try to replay them by making another request.

This may be due to a traffic log with details getting leaked or the request being sniffed in the network. So, it can happen.

Depending on the webhook, it can have an impact on your system especially if it integrates anything that deals with payments.

So, what can we do about it?

Timestamps

Stripe actually provides a neat way to overcome this challenge.

Within the webhook request, a timestamp is added as part of the signature sent in the request header.

Example - Request Header from Stripe:

Stripe-Signature:
t=1492774577,
v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd,
v0=6ffbb59b2300aae63f272406069a9788598b792a944a07aba816edb039989a39

by using the t value, the time the webhook was sent, we can then set an expiry interval for our webhook requests.

That means any events received beyond the time period we set, it would be considered invalid and should be rejected (ie t <= t + 5 minutes).

Keeping record

An even more robust solution is storing the webhooks that were received or a signature of it in a database or a cache (ie Redis).

That way every time you receive a webhook request, you can check the signature against the store to see if it was already received.

Conveniently, this implementation also handles duplicates if we ever receive them!

The downside of this approach is that the memory use may be large depending on the number of events you receive.

Conclusion

Before we conclude, let’s do a quick recap.

Recap:

The way we build applications today (using microservices) require some sort of way for us coordinate between the services

Webhook is technique that helps you coordinate between your application and a 3rd party service

Webhooks at its core are just http requests

The most common way to authenticate when using webhooks is via digital signatures using HS256

There are a few techniques to consider to keep your webhook endpoint secure in practice:

IP allow list

Authentication

Timestamp / Expiry interval

Encrypted connection (HTTPS)

That’s it! I hope you found this guide helpful.

If you did, please share this article with a friend or co-worker!

Appendix

HS256 explained

As mentioned, HMAC stands for Hash-based authentication code, and this allows us to establish trust between parties and systems.

HMAC requires the following:

A shared secret (or key)
Hashing algorithm (in our case it would be SHA-256)

Why does this work

It works mainly because of SHA-256.

Hashing algorithms, like SHA-256, are one-way cryptographic hash functions that would always generate the same output given the same inputs.

So, that means whenever we receive a webhook request, we can re-generate the same signature provided that we have the shared secret (which we do).

What you need to know about SHA-256

SHA-256 meets all criteria of what is considered a secure hashing function.

The criteria are:

Preimage resistance - The input cannot be determined from the output
Second preimage resistance (Weak collision resistance) - given m1, you cannot find m2 such that the hash of m1 is equal to the hash of m2, h(m1) = h(m2)
Collision resistance (Strong collision resistance) - It is difficult to find any m1 and m2 such that h(m1) = h(m2)

So, what are the chances of someone in a collision?

1 in 1^256 (This is a very small number, it is basically zero but not impossible!)

Note: Hashing functions are not to be confused with an encryption functions, you cannot reverse (or decrypt) a hash to determine its message.

📝 Helpful reference:

OKTA - HMAC

Also published here.