Pushpin — An Open Source Library That Turns REST APIs into Realtime APIs

An intro to evented APIs and how to build them with Pushpin

“Real-time” is becoming an omnipresent force in the modern tech stack. As consumers demand faster and more frequent data transactions, companies are increasingly investing in product infrastructure that accelerates these transactions. Though we’ve seen APIs become an economic and technological imperative, they are typically based on request-response style interactions, which limits their scope and effectiveness in the real-time arena.

Request-Response vs Event-Driven APIs

At its core, request–response is a message exchange pattern in which a requestor sends a request message to a replier system. The replier system receives and processes the request, and if all goes well, it returns a message in response. While this exchange format works well for more structured requests, it limits integrations to those where the expectant system has a clear idea what it wants from the other. These request-response style APIs, therefore, must follow the interaction script from the calling service.

In an event-driven architecture, applications integrate multiple services and products as equals based on event-driven interactions. These interactions are driven by event emitters, event consumers, and event channels, whereby the events, themselves, are typically significant ‘changes in state’ that are produced, published, propagated, detected, or consumed. This architectural pattern supports loose coupling amongst software components and services. The advantage is that an event emitter does not need to know the state of the consumer, who the consumer is, or how the event will be processed (if at all). It is a mechanism of pushing data through a persistent stream.

Evented API Solutions

In the tech ecosystem, there are a number of ways to approach data streaming and evented APIs. Some of the leading SAAS solutions include PubNub, Pusher, Kaazing, and Fanout — which each have their own pros/cons and ramp up investment. For the purposes of understanding the fundamentals of event-driven architecture, we’ll explore some open source software called Pushpin.

Pushpin

Pushpin’s primary value prop is that it is an open source solution that enables real-time push — a requisite of evented APIs (GitHub Repo). At its core, it is a reverse proxy server that makes it easy to implement WebSocket, HTTP streaming, and HTTP long-polling services. Structurally, Pushpin communicates with backend web applications using regular, short-lived HTTP requests.

This architecture provides a few core benefits:

• Backend languages can be written in any language and use any webserver.
• Data can be pushed via a simple HTTP POST request to Pushpin’s private control API.
• It is invisible to connected clients.
• It manages stateful elements by acting as the responsible party when requiring data from your backend server.
• Horizontally scalable by not requiring communication between Pushpin instances.
• It harnesses a publish-subscribe model for data transmission.
• It can act as both a proxy server and publish-subscribe broker.

Integrating Pushpin

From a more systemic perspective, there are a few ways you can integrate Pushpin into your stack. The most basic setup is to put Pushpin in front of a typical web service backend, where the backend publishes data directly to Pushpin. The web service itself might publish data in reaction to incoming requests, or there might be some kind of background process/job that publishes data.

Because Pushpin is a proxy server, it works with most API management systems — allowing you to do perform actual API development. For instance, you can chain proxies together, placing Pushpin in the front so your API management system isn’t subjected to long-lived connections. More importantly, Pushpin can translate WebSocket protocol to HTTP, allowing the API management system to operate on the translated data.

You can check out some sample apps built with Pushpin here.

The Future of Evented APIs

In some follow-up articles, I’ll discuss some of the unique features that we can see in evented APIs moving forward. These include event batching, salience filters, and a standard subscription interface. If you’re looking to play around with a real-time drop-in API proxy, then I highly recommend experimenting with Pushpin to get started.