Using Jest to Mock Elasticsearch

Since you opened this article, I assume you either use Elasticsearch or plan to.

Another assumption - you definitely found it a great idea to lock your functionality with some integration tests. And it really is!

The current company I’m working for has 90+% of code coverage with both unit and integration tests! I would recommend everyone cover their code base with tests since as one wise man said:

[Without unit tests] You’re not refactoring, you’re just changing shit.— Hamlet D’Arcy

Setup

Imagine you have a simple RESTful server with some logic that uses Elasticsearch. In the current showcase — CRUD server.

const Hapi = require('@hapi/hapi');
const Qs = require('qs');
const {createHandler} = require("./create/index.js");
const {readAllHandler, readHandler} = require("./read/index.js");
const {updateHandler} = require("./update/index.js");
const {deleteAllHandler, deleteHandler} = require("./delete/index.js");

const init = async () => {

    const server = Hapi.server({
    port: 3000,
    host: 'localhost',
    query: {
        parser: (query) => Qs.parse(query)
    }
    });

    server.route({
    method: 'POST',
    path: '/',
    handler: createHandler
    });

    server.route({
    method: 'GET',
    path: '/{id}',
    handler: readHandler
    });

    server.route({
    method: 'GET',
    path: '/',
    handler: readAllHandler
    });

    server.route({
    method: 'PATCH',
    path: '/{id}',
    handler: updateHandler
    });

    server.route({
    method: 'DELETE',
    path: '/{id}',
    handler: deleteHandler
    });

    server.route({
    method: 'DELETE',
    path: '/',
    handler: deleteAllHandler
    });

    await server.start();

    server.events.on('log', (event, tags) => {
    console.log({event}, {tags})
    if (tags.error) {
        console.log(`Server error: ${event.error ? event.error.message : 'unknown'}`);
    }
    });
    console.log('Server running on %s', server.info.uri);
};

process.on('unhandledRejection', (err) => {

    console.log(err);
    process.exit(1);
});

init();

Enter fullscreen mode Exit fullscreen mode. Now you need to cover the logic of each route with some tests to lock functionality and prevent business logic from being broken.

One obvious but not simple solution is to use Docker and spin up Elastic for tests every time.

However, is it worth it? I mean do you really want to have a longer spin-up time for the pipeline environment? Are there already-built solutions for that?

This plugin downloads and caches Elasticsearch binary when jest starts, then the plugin automatically starts Elastic on defined ports and tears it down when tests are done.

How to add tests?

I have an example of with/without jest tests setup in the pull request so you can compare all the changes. But now let’s go through it step by step.

1. Install additional modules

yarn add --dev jest @shelf/jest-elasticsearch @types/jest

Enter fullscreen mode Exit fullscreen mode

2. Add jest-config.js

touch jest.config.js

Enter fullscreen mode Exit fullscreen mode

module.exports = {
    preset: '@shelf/jest-elasticsearch',
    clearMocks: true,
    collectCoverage: true,
    coverageDirectory: "coverage",
    coverageProvider: "v8"
};

Enter fullscreen mode Exit fullscreen mode

Alternatively, you can generate jest config on your own using the CLI tool.

3. Add jest-es-config.js for the plugin

touch jest-es-config.js

Enter fullscreen mode Exit fullscreen mode

const {index} = require('./src/elastic.js');

module.exports = () => {
    return {
    esVersion: '8.4.0',
    clusterName: 'things-cluster',
    nodeName: 'things-node',
    port: 9200,
    indexes: [
        {
        name: index,
        body: {
            settings: {
            number_of_shards: '1',
            number_of_replicas: '1'
            },
            mappings: {
            dynamic: false,
            properties: {
                id: {
                type: 'keyword'
                },
                value: {
                type: 'integer'
                },
                type: {
                type: 'keyword'
                },
                name: {
                type: 'keyword'
                },
            }
            }
        }
        }
    ]
    };
};

Enter fullscreen mode Exit fullscreen mode

4. Extend package.json script to run tests

{
    "scripts": {
    "test": "jest"
    "serve": "node src/index.js"
    }
}

Enter fullscreen mode Exit fullscreen mode

5. Tune elastic client

const dotenv = require('dotenv')
dotenv.config()
const {Client} = require('@elastic/elasticsearch');

module.exports.client = new Client({
    node: process.env.NODE_ENV === 'test' ? 'http://localhost:9200' : process.env.ES_URL
})

module.exports.index = 'things'

Enter fullscreen mode Exit fullscreen mode

Add a condition that will use NODE_ENV to connect to local spined-up elastic whenever we are running tests.

Profit!

Now all the things are ready to write and run tests. All routes are fully covered and stored here:elastic-jest-example

As an example let’s cover creating function business logic.

const {ulid} = require('ulid');
const {client, index} = require("../elastic.js");

module.exports.createHandler = async (request, h) => {
    if (Object.keys(request.payload))
    try {
        const res = await this.create(request.payload)
        return h.response(res).code(200);
    } catch (e) {
        console.log({e})
        return h.response({e}).code(400);
    }
}

// let's cover this function with some tests
module.exports.create = async (entity) => {
    const {
    type,
    value,
    name,
    } = entity;

    const document = {
    id: ulid(),
    type: type.trim().toLowerCase(),
    value: +value.toFixed(0),
    name: name.trim()
    }

    await client.index({
    index,
    document
    });
    return document.id
}

Enter fullscreen mode Exit fullscreen mode

Create a test file and add a couple of statements.

touch  src/create/index.test.js

Enter fullscreen mode Exit fullscreen mode

const {create} = require("./index.js");
const {client, index} = require("../elastic");

describe('#create', () => {

// clear elastic every time before running it the statement.
// It's really important since each test would be idempotent.
    beforeEach(async () => {
    await client.deleteByQuery({
        index,
        query: {
        match_all: {}
        }
    })
    await client.indices.refresh({index})
    })

    it('should insert data', async () => {
    expect.assertions(3);
    const res = await create({type: 'some', value: 100, name: 'jacket'})
    await client.indices.refresh();
    const data = await client.search({
        index,
        query: {
        match: {
            "id": res
        }
        }
    })

    expect(res).toEqual(expect.any(String))
    expect(res).toHaveLength(26);
    expect(data.hits.hits[0]._source).toEqual({
        "id": res,
        "name": "jacket",
        "type": "some",
        "value": 100
        }
    );
    })

    it('should insert and process the inserted fields', async () => {
    const res = await create({type: 'UPPERCASE', value: 25.99, name: ' spaces '})
    await client.indices.refresh();
    const data = await client.search({
        index,
        query: {
        match: {
            "id": res
        }
        }
    })
    expect(data.hits.hits[0]._source).toEqual({
        "id": res,
        "name": "spaces",
        "type": "uppercase",
        "value": 26
        }
    );
    })
});

Enter fullscreen mode Exit fullscreen mode

A basic testing flow for each business logic function can be simply described like this:

insert data-> run tested function -> check outputs -> clear data -> repeat

Data insertion/deletion can be improved by unifying them into helpers and using additional mooching libs.

The elastic teardown is managed by @shelf/jest-elasticsearch lib itself.

One more good convention to follow is to cover each function you are testing with describeblock so that later you can easily run a specific test with IDE helper without rerunning the whole suite:

Resources

Now you know how to test your Elasticsearch queries using jest jest-elasticsearch

Also, you have a small blueprint repo with ready to use setup.

Hope this article will help you set up and test your elastic project!

Want to connect?

Follow me on Twitter!

Also published [here.](https://Originally published here)