Connectors: a modular and middleware-based access to an API

During our work on crudl, a backend agnostic admin interface, we developed connectors in order to access different and heterogenous APIs in a unified and modular way. Since then, I have used connectors in other projects as well and grew increasingly more convinced that the underlying concepts are valid and with potential for further development. In this article, I will introduce the concepts and demonstrate how you can employ connectors to implement an OAuth 2 token refreshing functionality that is transparent to the application.

So what is a connector? A connector represents a single API endpoint and provides create, read, update, delete methods to access and manipulate the resources at this endpoint. The crud methods accept requests and return promises. These promises either resolve to http responses or reject with an error. Because the crud methods return promises, they can be chained. The chaining of connectors allows for a high degree of modularity: each connector in the chain provides a specific functionality that is achieved by amending requests and processing responses and errors.

The simplest configuration of a connector chain is a frontend-backend pair. The backend connector uses the passed requests to create http calls that the API understands. In a fashion similar to Express, the frontend connector’s use method extends the basic pair by including middleware connectors in the chain.

Note: whereas backend and middleware connectors’ crud methods resolve to http responses, the frontend connector methods resolve to the response data. This is an important feature that keeps the response format transparent to the software layers above connectors.

It is the usage of middleware that makes connectors a powerful tool. Have a look on this small yet fully functioning example:

Example 1: Basic setup

The following code shows a functioning, ready-to-run connectors setup. You can clone this example from my github repo (example reqres) and try it out.

<a href="https://medium.com/media/d324135e29888a12d4f3babdc1cf36bd/href">https://medium.com/media/d324135e29888a12d4f3babdc1cf36bd/href</a>

Let’s break down what’s happening in the code. After importing the main functions from the connectors package and a pair of middleware creators, we define a baseURL common to all our connectors (we’re using the public reqres API in this example).

We then define a connector creator function base(urlPath: String) which creates the basic frontend-backend connector pair and extends it with two middleware connectors:

• crudlToHttp(mapping: Object) creates a middleware connector that translates the crud methods to http methods. Without any arguments passed to it, it uses the default mapping: create → POST, read → GET, update → PATCH, and delete → DELETE.
• url(urlPath: String) creates a middleware connector that adds an url attribute to the request objects passed to it. The backend connector requires this attribute in order to create a valid http request. The url middleware understand express-like URLs with parameters. For example users/:id.

We then create two connectors: users to access the list endpoint and user to access the detail endpoint. We use the users connector to create a new user (line 16) and the detail connector user to update a single user (line 20).

Note that the detail connector is using a parametrized URL. In order to resolve the :id parameter of the user URL we need to pass that parameter down the connector chain so that it reaches the url middleware. For that purpose we use the parametrization feature of the frontend connector. Calling a frontend connector as a function with the parameters as arguments creates a new frontend connector. Whenever one of its crud methods is called, this new frontend connector adds these parameters to the request object’s params attribute. It is the responsibility of the middleware connectors to consume the right amount of parameters that they require and pass the rest further down the stream. Partial parametrization is also possible and you can do for example the following:

<a href="https://medium.com/media/e466e7c037694676e44dbb6450aee17b/href">https://medium.com/media/e466e7c037694676e44dbb6450aee17b/href</a>

This example showed how to create connectors and extend them with middleware. The next example should reveal the potential behind these concepts.

Example 2: Transparent Token Refreshing using Connectors

The context. We have an app that connects to an API with OAuth 2 authentication. In order to access the resources, the app must provide an access token with each API call. Every access token expires after a while and the app must request a new one from the server using a long-live refresh token. The access and refresh tokens are stored in a globally accessible object called session (placed for example in a redux store). If the tokens are present the user is considered logged-in, otherwise the user is considered logged-out and the app redirects the user to a login page.

The task. We want to implement the refresh token logic such that it is completely transparent to the app. The app should access the resources oblivious of access and refresh tokens. Refreshing the access token should happen in the background as a part of the app’s crud request. If refreshing of the token fails, the user should be simply logged out.

The approach. We will implement the refresh token logic as a middleware connector executing the following algorithm: The middleware connector includes the current access token in the request and passes it to the next connector. If this request results in a 401 error (unauthorized), it tries to refresh the token. If the refreshing was successful, it repeats the request with the new access token and returns the response. If refreshing the token failed, the middleware connector clears the session and so logs out the user.

Implementation. The middleware will be called accessToken, but before I show you its implementation, let’s have a look how it will be used. The following code snippet creates three connectors. A token connector to a non-protected part of the API for obtaining and refreshing the access tokens, and two connectors (posts and comments) accessing protected resources.

<a href="https://medium.com/media/9d44d5df23f7d97693bf0680c46e4cb0/href">https://medium.com/media/9d44d5df23f7d97693bf0680c46e4cb0/href</a>

Note how modularly we apply the middleware. All connectors in this example use the url and crudToHttp middleware, but only those connectors that access the protected parts use the accessToken middleware. Similarly, only the connectors accessing the non-protected resources use the basicAuth middleware. Figure 4. shows a graphical representation of this particular configuration of the connector chains.

Finally, here’s is the complete code for the accessToken middleware:

<a href="https://medium.com/media/2a0705dce835bc630ed9da7d76150787/href">https://medium.com/media/2a0705dce835bc630ed9da7d76150787/href</a>

When we call the frontend connector’s use() function, the accessToken function will be invoked with the next connector in the chain as its parameter. The middleware function creates a new connector that provides all four crud methods wrapped in the refreshOn401 function that executes the automatic refresh token logic and passes requests to the next connector in the chain.

Note that the functions refreshToken and clearSession must also return promises for the code to work. In a React/Redux application, these two functions would be asynchronous actions that update the redux store and the function getAccessToken would be a selector for obtaining the access token from the store.

That’s all! The application can now create, read, update, or delete posts and comments without caring for access tokens and their refreshment. A simple call like this posts.create({ data: { title: '...', text: '...' } }) creates a new posts even if the app’s access token expired. The accessToken middleware refreshes the token in the background and repeats the request.

Summary

Here’s a summary of the key concepts and features.

A connector connects to a single API endpoint, provides create, read, update, and delete methods which accept request objects and return promises. A connector can be chained with other connectors. There are three classes of connectors: frontend, middleware, and backend connectors. The most basic chain of connectors is a frontend-backend pair. This pair can be extended by middleware.

A frontend connector is the head of a connector chain and the app’s entry point to an API. Besides the crud methods, a frontend connector provides a use method to insert middleware in the chain. Frontend connectors are the only connectors whose methods do not return promises resolving to http responses. Instead, they return promises that resolve to response data. This makes usage of connectors intuitive and keeps the response format transparent to the software layers above. You can send parameters to the whole connector chain by calling the frontend connector as a function and passing the parameters as arguments. This allows for a code like this: post(postID).delete(), which is a shortcut for post.delete({ params: [ postID ] }).

A backend connector takes the passed requests and translates them into http requests. Currently we’re using axios for backend connector.

A middleware connector is a connector in the chain between a frontend and backend connector. To include a middleware connector in the chain you call the frontend’s connector use method which takes a connector creator as its argument. This middleware creator function, with the signature (next: Connector) => <Connector>, builds a new middleware connector. The next argument is the immediate successor in the chain. A middleware connector may implement one or more crud methods. Those methods left unimplemented will be replaced by a pass-through function such as (req) => next.create(req).

What’s next?

The connectors package is still young and needs to grow and mature a little. I’d like to see it develop into a concise, well tested, well documented, easy to debug package that provides a robust platform for a variety of middleware. I fantasize about a community of github repos providing middleware connectors for accessing various APIs (REST or GraphQL) and providing services such as session management, error processing, frontend caching, etc. The package is open source and we’d be happy to receive your input, be it ideas, comments, or code.