Testing the code you write is an important step in the process of software engineering. It ensures that your software works as expected and reduces the risk of shipping bugs and vulnerabilities to production. Automated tests, in particular, play an important role when it comes to testing frequently and consistently. Continuous integration makes them even more powerful. In this blog post, I'll show you an architecture for testing Node.js REST APIs that use a in the background. There are some things you should consider in this scenario that we'll talk about. database You'll see how to separate and organize your application's components in a way that you can test them independently. Therefore, we'll use . On the one hand, we set up a test environment in which we run our tests against a . two different approaches test database On the other hand, we using so that we can run them in an environment in which we have no access to a database. mock the database layer mock functions We'll start by writing unit tests for testing single components of our application. In the next step, we combine those components and test them using integration tests. Last but not least, we set up a CI/CD pipeline using GitHub actions and run the tests on each push that is made. Note that this is not a guide on how testing in general works. There are thousands of articles about frameworks like Jest, Mocha, Supertest, and so on. This is more of a guide on how to prepare your Node application and test environment in a way that you can write tests effortlessly and efficiently . There's also an example repo on . You should definitely check it out. with or without a database connection GitHub Disclamer: I know there really is no right or wrong when it comes to the architecture. The following is my prefered one. Let's start with the tools we'll use. Most of them should be familiar to you: : Language Typescript : Server Express : Database Postgres : & & Testing Jest Supertest Chai : CI/CD GitHub Actions : Containerization Docker One benefit of this architecture is that you can use it with other databases than Postgres too, like for example. In this architecture, we don't use any kind of ORM. Moreover, you can replace Jest with if that is your desired testing framework. MySQL Mocha Application Architecture The architecture of our application looks roughly like this: node-api ├── api │ ├── components │ │ ├── user | | │ ├── tests // Tests for each component | | | │ ├── http.spec.ts | | | │ ├── mock.spec.ts | | │ | └── repo.spec.ts | | │ ├── controller.ts | | │ ├── dto.ts | | │ ├── repository.ts | | │ └── routes.ts │ └── server.ts ├── factories // Factories to setup tests | ├── abs.factory.ts | ├── http.factory.ts | └── repo.factory.ts └── app.ts Note: The example repo contains some more code. Each component consists of the following four files: : HTTP Handler controller.ts : Data Transfer Object ( ) dto.ts more : Database Layer repository.ts : HTTP Routing routes.ts The directory includes the tests of the according component. If you want to read more about this architecture, check out article of mine. tests this Config We'll start by creating an file that contains the secret environment variables for testing. The npm Postgres package uses them automatically when establishing a new database connection. All we have to do is to make sure that they are loaded using . .env.test dotenv NODE_PORT=0 NODE_ENV=test PGHOST=localhost PGUSER=root PGPASSWORD=mypassword PGDATABASE=nodejs_test PGPORT=5432 Setting lets Node choose the first randomly available port that it finds. This can be useful if you run multiple instances of an HTTP server during testing. You can also set a fixed value other than here. Using we provide the name of our test database. NODE_PORT=0 0 PGDATABASE Next, we set up Jest. The config in looks as follows: jest.config.js module.exports = { preset: "ts-jest", testEnvironment: "node", roots: ["src"], setupFiles: ["<rootDir>/setup-jest.js"], } And like this: setup-jest.js require("dotenv").config({ path: ".env.test", }) This snippet ensures that the appropriate environment variables are loaded from the provided file before running the tests. .env Testing With Database Let's start with the assumption that we have a test database that we can use. This might be one in a GitHub Actions CI/CD pipeline for example. Later on, I'll show you how to test your application without a database connection. 3 rules In the beginning, I said that there are some important things we want to consider that make life much easier when testing Node APIs: Separate the database layer Outsource the database connection initialization Outsource the HTTP server initialization What do I mean by that? Separate the database layer. You should have your own layer, separated from your business logic, that takes care of the communication with the database. In the example Git repo, you can find this layer in a component's file. repository.ts This allows us to easily mock a layer's methods when we have no database available for testing. Moreover, it's easier to replace your database system with another one. export class UserRepository { readAll(): Promise<IUser[]> { return new Promise((resolve, reject) => { client.query<IUser>("SELECT * FROM users", (err, res) => { if (err) { Logger.error(err.message) reject("Failed to fetch users!") } else resolve(res.rows) }) }) } Outsource the database connection initialization. You should already know that when writing tests against a database, you do not run them against the production one. Instead, you set up a test database. Otherwise, you run the risk of messing up your production data. Most of the time, your application connects to the database in a startup script, like . After the connection is established, you start the HTTP server. index.js That's what we want to do in our test setup too. This way, we can connect to the database and disconnect from it gracefully before and after each test case. Outsource the HTTP server initialization. It's a good practice, whether you use a database or not, to start the HTTP server from inside your tests. Just as we do for the database connection, we create a new HTTP server before and stop it after each test case. This might look as follows: (you'll see the concrete implementation later on) describe("Component Test", () => { beforeEach(() => { // Connect to db pool && start Express Server }); afterEach(() => { // Release db pool client && stop Express Server }); afterAll(() => { // End db pool }); In particular, the execution order is: Connect to the database pool Run the SQL seed (create tables) Start the Express server Run the tests Shutdown the Express server & release db pool client Repeat steps 1 - 5 for each test case and close the pool at the end Tests Each component consists of two test files: repo.spec.ts http.spec.ts Both of them make use of so-called which prepares the test setup. You'll see their implementation in the next chapter. TestFactories Note: If you have a look at the example Git repo, you'll see that there are two more: mock.spec.ts and dto.spec.ts. Former one is discussed later on. The latter is not covered in this article. repo.spec.ts A repository is an additional abstract layer that is responsible for interacting with the database like reading and inserting new data. That layer is what we test in here. Since a database is required in this case, a new pool client is created to connect to the database using the before each test case. And it is released, right after the test case is completed. In the end, when all test cases are finished, the pool connection is closed. RepoTestFactory Example on GitHub describe("User component (REPO)", () => { const factory: RepoTestFactory = new RepoTestFactory() const dummyUser = new UserDTO("john@doe.com", "johndoe") // Connect to pool beforeEach(done => { factory.prepare(done) }) // Release pool client afterEach(() => { factory.closeEach() }) // End pool afterAll(done => { factory.close(done) }) test("create user", () => { const repo: UserRepository = new UserRepository() repo.create(dummyUser).then(user => { expect(user).to.be.an("object") expect(user.id).eq(1) expect(user.email).eq(dummyUser.email) expect(user.username).eq(dummyUser.username) }) }) }) http.spec.ts Here, we test the integration of the user component's routes, controller, and repository. Before each test case, a new pool client is created just as we did above. In addition, a new Express server is started using the . In the end, both are closed again. HttpTestFactory Example on GitHub describe("User component (HTTP)", () => { const factory: HttpTestFactory = new HttpTestFactory() const dummyUser = new UserDTO("john@doe.com", "johndoe") // Connect to pool && start Express Server beforeEach(done => { factory.prepare(done) }) // Release pool client && stop Express Server afterEach(done => { factory.closeEach(done) }) // End pool afterAll(done => { factory.close(done) }) test("create user", async () => { const postRes = await factory.app .post("/users") .send(dummyUser) .expect(201) .expect("Content-Type", /json/) const postResUser: IUser = postRes.body expect(postResUser).to.be.an("object") expect(postResUser.id).eq(1) expect(postResUser.email).eq(dummyUser.email) expect(postResUser.username).eq(dummyUser.username) }) }) Factories The test factories are actually the heart of our tests. They are responsible for setting up and preparing the environment for each test case. That includes: Dropping & creating all db tables Initializing the db connection Initializing the HTTP server Closing both of them again There are four in total: , , , and . Each of them has its own Typescript class. The last one is discussed in the chapter "Testing without database". factories AbsTestFactory RepoTestFactory HttpTestFactory MockTestFactory AbsTestFactory The first one is an abstract base class that is implemented by the other three. It includes, among others, a method for connecting to the database pool and one for disconnecting from it. AbsTestFactory export abstract class AbsTestFactory implements ITestFactory { private poolClient: PoolClient private seed = readFileSync( join(__dirname, "../../db/scripts/create-tables.sql"), { encoding: "utf-8", } ) abstract prepareEach(cb: (err?: Error) => void): void abstract closeEach(cb: (err?: Error) => void): void protected connectPool(cb: (err?: Error) => void) { pool .connect() .then(poolClient => { this.poolClient = poolClient this.poolClient.query(this.seed, cb) }) .catch(cb) } protected releasePoolClient() { this.poolClient.release(true) } protected endPool(cb: (err?: Error) => void) { pool.end(cb) } } Using the script, the factory all the tables after the connection is established: create-tables.sql drops and recreates DROP TABLE IF EXISTS users; CREATE TABLE users ( id SERIAL PRIMARY KEY, email VARCHAR(50) UNIQUE NOT NULL, username VARCHAR(30) UNIQUE NOT NULL, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); RepoTestFactory The is used by each component's repository test ( ) that you just saw above. All it does is use the parent class to connect to the database. RepoTestFactory repo.spec.ts AbsTestFactory export class RepoTestFactory extends AbsTestFactory { prepareEach(cb: (err?: Error) => void) { this.connectPool(cb) } closeEach() { this.releasePoolClient() } closeAll(cb: (err?: Error) => void) { this.endPool(cb) } } The methods , , and , are called for each test case in the Jest , , and lifecycle. prepareEach closeEach closeAll beforeEach afterEach afterAll HttpTestFactory The last one, , is used by each component's HTTP test ( ). Just like , it uses the parent class for the database connection. Furthermore, it initializes the Express server. HttpTestFactory http.spec.ts RepoTestFactory export class HttpTestFactory extends AbsTestFactory { private readonly server: Server = new Server() private readonly http: HttpServer = createServer(this.server.app) get app() { return supertest(this.server.app) } prepareEach(cb: (err?: Error) => void) { this.connectPool(err => { if (err) return cb(err) this.http.listen(process.env.NODE_PORT, cb) }) } closeEach(cb: (err?: Error) => void) { this.http.close(err => { this.releasePoolClient() cb(err) }) } closeAll(cb: (err?: Error) => void) { this.endPool(cb) } } Rewind Let's jump back to the and test files from above. In both of them, we used the factories' method before each and its method after right each test case. repo.spec.ts http.spec.ts prepareEach afterEach The method is called at the very end of the test file. As you have just seen, depending on the type of factory, we establish the database connection and start the HTTP server if needed. closeAll describe("Component Test", () => { beforeEach(done => { factory.prepareEach(done) }) afterEach(() => { factory.closeEach() }) afterAll(done => { factory.closeAll(done) }) }) One important thing you should keep in mind is that for each test case that uses the database, the factory drops all the tables and recreates them using the provided SQL script afterward. This way we have a clean database with empty tables in each test case. Testing Without Database So far, we have run our tests against a test database, but what if we have no access to a database? In this case, we need to our database layer implementation ( ), which is quite easy if you have separated it from the business logic, as I recommended in rule #1. mock repository.ts With mocks, the layer does not depend on an external data source anymore. Instead, we provide a custom implementation for the class and each of its methods. Be aware that this does not affect the behavior of our since it does not care about where the data comes from. controller Example on GitHub const dummyUser: IUser = { id: 1, email: "john@doe.com", username: "john", created_at: new Date(), } // Mock methods const mockReadAll = jest.fn().mockResolvedValue([dummyUser]) // Mock repository jest.mock("../repository", () => ({ UserRepository: jest.fn().mockImplementation(() => ({ readAll: mockReadAll, })), })) After mocking the database layer, we can write our tests as usual. Using , we make sure that our custom method implementation has been called. toHaveBeenCalledTimes() describe("User component (MOCK)", () => { const factory: MockTestFactory = new MockTestFactory() // Start Express Server beforeEach(done => { factory.prepareEach(done) }) // Stop Express Server afterEach(done => { factory.closeEach(done) }) test("get users", async () => { const getRes = await factory.app .get("/users") .expect(200) .expect("Content-Type", /json/) const getResUsers: IUser[] = getRes.body cExpect(getResUsers).to.be.an("array") cExpect(getResUsers.length).eq(1) const getResUser = getResUsers[0] cExpect(getResUser).to.be.an("object") cExpect(getResUser.id).eq(dummyUser.id) cExpect(getResUser.email).eq(dummyUser.email) cExpect(getResUser.username).eq(dummyUser.username) expect(mockReadAll).toHaveBeenCalledTimes(1) }) }) Note: cExpect is a named import from the "chai" package. MockTestFactory Just as we did in the other tests files, we use a test factory here as well. All the does is run a new Express HTTP instance. It establish a database connection since we mock the database layer. MockTestFactory does not export class MockTestFactory extends AbsTestFactory { private readonly server: Server = new Server() private readonly http: HttpServer = createServer(this.server.app) get app() { return supertest(this.server.app) } prepareEach(cb: (err?: Error) => void) { this.http.listen(process.env.NODE_PORT, cb) } closeEach(cb: (err?: Error) => void) { this.http.close(cb) } } One drawback we have using this approach is that the layer ( ) is not tested at all because we overwrite it. Nevertheless, we can still test the rest of our application, like the business logic for example. Great! repository.ts Running Using the commands below, we can run the tests with or without a database. Depending on the scenario, the files we do not want to test are excluded from execution. { "test:db": "jest --testPathIgnorePatterns mock.spec.ts", "test:mock": "jest --testPathIgnorePatterns \"(repo|http).spec.ts\"" } GitHub Actions The final step is to create a CI/CD pipeline using GitHub actions that runs our tests. The according yaml file is available . There's also a very good published on GitHub. here tutorial You can decide whether to run the tests against a test database or use the mocked data layer. I decided to go with the former. When running the pipeline with a test database, we need to make sure that we set the correct environment variables for it. you can find a test run. Here Last Words My last tip is to have a look at the example repository on and to read it carefully There are some more tests and code snippets that I did not cover in this article. Moreover, check out the links below. Happy coding! GitHub Further resources ES6 Class Mocks Database seeding in Node.js How I structure my Node.js REST APIs Creating PostgreSQL service containers
