Standardizing HTTP API testing

This blog post is actually a draft for a standard operating procedure for my software development and consulting business. I’ve come across the task of writing tests for HTTP APIs for three or four times in the last couple of months. I’ve tried multiple ways of writing automated tests and this is the method I’ve converged on. TypeScript, Jest, and supertest appear to work well together and are sufficient to implement concise tests.

Background and the need for standardization

Years ago most of my programming for the web was limited to PHP. Tests on those project were done by using PHPUnit and Guzzle. Later we also used Behat for more readable tests. After Docker became popular the PHP monolith evolved into an application built with multiple programming languages. Some small services are now written in Node.js, and new data analytics code is mostly in Python.

At first, it seemed obvious to write API level tests in the same language as the server. But after comparing solutions for Node.js, PHP and Python it became evident that quality and approaches vary between languages.

Besides variance in quality, the problem was also time lost to learning each test framework. Investing time to learn Python was worth it because we can now use Pandas and Scipy to do analytics. And Jupyter is awesome for prototyping. In contrast, learning to test HTTP APIs in each language has no value. There is also no technical reason to program a web service and accompanying tests using the same language.

There are multiple benefits to having a standard procedure for a task like HTTP API tests: you write a lot of tests with the same framework. After you learn the basics you can focus on higher level problems such as organizing tests, determining what to cover and testing more complicated APIs. Each time you create a new web service, you don’t waste time on Googling phrases like “what’s the best way to test REST API?”. Because of decision to standardize this piece of my workflow, I’m saving brain bandwidth for other decisions.

My standard test stack

My experience with TypeScript has been overwhelmingly positive for the last couple of months. Most of the time I’ve been using it on the frontend with Angular. But it performs equally well on the backend too. Available tooling and type definitions make development with Express a breeze. Autocomplete functionality in IDEs such as VS Code is superb in comparison to plain JavaScript. Type checking helps a lot especially when refactoring. It supports modern features from ES2017 such as async/await. I used to use Karma as a testing framework but finding a good combination of plugins and writing configuration was a nightmare. Jest is a pleasant improvement. It works out of the box and has the common APIs you might already be familiar with: describe, it, expect. Another important ingredient for testing HTTP APIs is a way of making requests. Superagent has very good documentation and it works well with TypeScript. I like its chainable syntax.

Example of testing a simple GET request

import request = require('supertest');

const BASE_URL = 'http://my-service:3000';

describe('Hello world', () => {    it('Should say "hello world"', async() => {        const response = await request(BASE_URL)            .get('/')            .expect(200);        expect(response.text).toBe("hello world");    });});

Function describe groups together tests for a specific feature. Function it defines a test. In this particular case the test is an asynchronous function. It could also be a normal function or a function returning a Promise. In the next four lines we see the elegance of async/await syntax. The execution is still asynchronous but the syntax appears to be very linear. This aligns the syntax with our mental model of consecutive steps. The line .expect(200) asserts that the response code is indeed 200. And finally we also check the contents of the response.

POST requests

Sending data with POST and PUT requests looks like this:

import request = require('supertest');

const BASE_URL = 'http://my-service:3000';

describe('POST /news', () => {    it('Should save and return back the news', async() => {        const response = await request(BASE_URL)            .post('/news')            .send({'title': 'Breaking news!'})            .expect(201);

        expect(response.body).toEqual({'id': 1, 'title': 'Breaking news!'});    });});

Instead of get(url) we used post(url) and added a call to send(data) to send our JSON.

Authorization and other headers

Headers can be set using set('header-name', 'value') as seen in the following example:

import request = require('supertest');

const BASE_URL = 'http://my-service:3000';

describe('Authorization test', () => {    it('Should respond with 401 if token invalid', async () => {        await request(BASE_URL)            .post('/admin/news')            .set('Authorization', 'Bearer invalid')            .send({})            .expect(401)            .expect('WWW-Authenticate', /^Bearer/);    });});

I’m hoping this article will help you to set up tests quicker. Complete code is available at https://gitlab.com/mdrolc/testing-http-apis There you will find a simple service and corresponding tests.

Originally published at https://drola.si/post/2019-02-19-testing-http-apis on February 19, 2019.