paint-brush
The State of Webhooks in 2023by@tomhacohen
1,336 reads
1,336 reads

The State of Webhooks in 2023

by Tom HacohenOctober 28th, 2023
Read on Terminal Reader
Read this story w/o Javascript

Too Long; Didn't Read

Webhooks are now a core component in driving real-time event based workflows across platforms. We looked at over 100 of the top API providers, examining how they've embraced and implemented webhooks. Are most leading API providers on board with best practices? How many retries do these messages generally need before they successfully land in their destined applications?
featured image - The State of Webhooks in 2023
Tom Hacohen HackerNoon profile picture


As software continues to eat the world, an increasing emphasis on real-time information, seamless integrations, and automation has propelled webhooks to the forefront of modern application architectures. Webhooks, are now a core component in driving real-time event based workflows across platforms. You can read the full report here.


Special thanks to Ojus Save from Zoom, Judy Vander Sluis from Intuit, Sharon Yelenik from Cloudinary, Sarah Edwards from Github, and Kanad Gupta from ReadMe for collaborating with us on our webhook documentation reviews. It really helped us understand how people think about documenting their webook APIs and informed a lot of our decisions when creating this report.


In this comprehensive report on the State of Webhooks in 2023, we looked at over 100 of the top API providers, examining how they've embraced and implemented webhooks, and analyzing the diverse ways these implementations have taken form. Are most leading API providers on board with best practices? Beyond mere adoption, how have they optimized, secured, and enriched their webhook offerings to cater to the needs of today’s developers and businesses?


Recognizing that ground truths often emerge in real-world use-cases, we've tapped into our own customer base, compiling an intriguing cache of statistics that shed light on the realities of webhook deliveries. How often do they falter in the wild? How many retries do these messages generally need before they successfully land in their destined applications? These firsthand statistics not only offer a clear picture of current delivery success rates but also demonstrate the value of implementing best practices.


Generally, we found webhook adoption is high at 83%. However, the adoption of most best practices is lagging.

Retries

Retries involve re-sending a webhook if an attempt fails. They are crucial for a reliable webhook service because temporary network issues, server downtimes, or other transient errors can impede immediate data delivery.


67% of services offered automatic retries. The most common amount of retries offered is 5 with most offering between 3- 10 retries. Around 10% of the services stated they retried failed messages, but did not provide any information about the retry schedule itself.


Distribution of the number of retry attempts offered

Retries: Exponential Backoff

Webhook retries use exponential backoff to efficiently handle failures without overwhelming the receiving server.


By progressively increasing the wait time between retries, it reduces the risk of exacerbating potential server issues and provides a more adaptive approach to handling transient failures.


25/83 providers specified that their retry schedule follows an exponential backoff.



In conclusion, webhooks are being adopted by most API providers, but they mostly fail to implement best practices. Even the ones that do implement most best practices do so in different ways. The space is so fragmented the only providers with similar implementations were those that did not implement best practices at all. Hopefully, this report will spark an increase in adoption of webhook best practices to help improve the developer experience around webhooks.

Manual Retries

Being able to retry messages manually speeds up troubleshooting. It is faster to trigger a retry immediately instead of waiting for the next automatic retry. 12/83 providers specified that retries could be triggered manually. This was the least adopted best practice.

Logging

For testing, troubleshooting, and debugging purposes, a log of webhook delivery attempts is extremely useful. This was the second least adopted functionality at 23% adoption.

Event Types

Organizing the events offered to webhook consumers into event types lets users choose which events they want to receive webhooks for and cuts down on the number of unnecessary and irrelevant messages being sent.


93% of providers offered event types. This is the most widely adopted best practice which probably stems from the fact that events are the core value of webhooks.

Message Authentication

Giving users a way to authenticate the origin and content of a webhook message is essential for ensuring that data has not been tampered with during transit and confirms it originates from a trusted source


The best practice is to use HMACSHA256 signatures that has the payload, timestamp, and message ID.



45 of 83 webhook providers included a timestamp. The timestamp is critical for preventing replay attacks.

Documentation

Documenting a webhook service well can save users time and spare them a headache.


Having sample code makes developers lives easier. While only 43% offered code samples, the inclusion of code samples correlated heavily with using HMACSHA256 signatures.


Documentation

Testing

The best webhook documentation gives guidance on how to test their webhook implementations. Common guidance includes how to test webhooks locally (mostly using ngrok), listing various tools for spinning up endpoints to receive test messages, and providing the ability to send test events.


There is a high correlation between having code samples and providing guidance and tooling to test webhooks.


72% of those with code samples in their docs also provided testing guidance.

Delivery

In the full report, we also present some internal data from Svix to to see how often messages fail, how often retries succeed, average response times, and average payload sizes of webhook messages.


Read the Full Report



For more content like this, make sure to follow us on Twitter, Github or RSS for the latest updates for the Svix webhook service, or join the discussion on our community Slack.