Automated Testing with GitHub Actions

In this article, we want to explore how to automatically test our code with GitHub Actions when pushing updates to our remote repository.

We will look at a minimal example to discover the main building blocks of GitHub Actions and learn how to set it up.

Prerequisites

Some basic knowledge of programming is assumed, as well as an understanding of the concept of Continuous Integration and Continuous Delivery (CI/CD).

The code examples are in JavaScript but should be understandable without JS knowledge.

The concepts and ideas are transferrable to other programming languages and frameworks.

What is GitHub Actions?

GitHub Actions is a CI/CD platform that automates building, testing, and deploying code.

It allows us to create workflows that are triggered by events. An example is a workflow that deploys the code to production when a pull request is merged into the main branch.

Another example would be a workflow to run the test suite when a pull request is opened or updated which is what we will look at in this article. The repository can be set up to only allow certain activities after a workflow passes successfully, e.g., merging a pull request could be only allowed after workflows for testing, lining, and formatting the code have passed.

GitHub Actions even goes one step further and allow for the creation of workflows that extend beyond the code. It is possible, for example, to create a workflow that automatically adds an appropriate label to issues that are being created in the repository.

Example Project

To play around with GitHub Actions, we will look at a very simple example in Node.js.

We will write a basic function that adds two numbers and implement a unit test for that function using Jest. We will then create a GitHub Actions workflow to automatically run the test suite on every PR and every push to the main branch.

You can head over to the demo repository on GitHub to see the code and the repository setup.

Since this is a very hands-on demo, it could be worth it to fork the repository, try the steps below for yourself and extend the code for more complex scenarios.

However, just reading the article should give you a good overview and provide you with a basic understanding of GitHub Actions.

Our setup is quite simple, a Node.js project with Jest as a dev dependency for unit testing.

We will start by creating a file app.js with a simple sum function:

function sum(a, b) {
  return a + b;
}
module.exports = { sum };

Next, we will create a file app.test.js with one test case for the sum function:

const sum = require("./app").sum;

test("add two numbers", () => {
  expect(sum(3, 5)).toBe(8);
});

Finally, let’s add a test script to our package.json:

{
	...
  "scripts": {
    ...
    "test": "jest"
  },
  ...
}

We can run our test suite from the command line via npm test.

While most developers probably run the test suite before pushing any changes to the code base, that step can be easily forgotten. Furthermore, running the tests in larger projects can take quite a long time and we might be tempted to skip the tests when having to push something urgently.

Therefore, we want to make sure that the tests run automatically with every change to the codebase.

Let us look at how GitHub Actions can help achieve that.

Workflow for Automated Testing

We have a very basic app and a test suite that we can run from the command line.

Now, let’s automate that step and create a GitHub Actions workflow for testing the code.

The goal is to have the test suite run automatically every time we push a change to the main branch or create a pull request.

Building the workflow

To add workflows to a GitHub repository, we create a top-level folder .github with a subfolder workflows. In there, we add a YAML file for every workflow we want to have. In our case, that will be only one file test.yml:

name: test
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: "16"
      - run: npm ci
      - run: npm test

Let’s look at the building blocks of that file in more detail.

name: test

👉 Quite straightforward: The name of our workflow.

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

👉 The on block describes which events trigger the workflow. In this case, we want our test workflow to run on any pushes to the branch main and when a pull request into main is created or updated.

jobs:
  test:
    runs-on: ubuntu-latest

👉 The jobs block lists all jobs the workflow consists of. A job is executed on a runner and consists of one or more steps. In our case, we have only one job called test which runs on an Ubuntu runner.

GitHub provides a set of runners to choose from, e.g. Ubuntu, macOS and Windows Server.

More complex workflows consist of multiple jobs, e.g. a build-and-deploy workflow could contain jobs to first validate the code, then build it and then deploy it.

When there are multiple jobs in a workflow, they are executed in parallel by default but it is possible to run jobs sequentially by defining dependencies.

steps:
  - uses: actions/checkout@v3
  - uses: actions/setup-node@v3
    with:
      node-version: "16"
  - run: npm ci
  - run: npm test

👉 The steps block describes exactly what our test job does.

Each step can either run a reusable action or a custom script.

The GitHub marketplace provides numerous actions for different environments and use cases.

To not reinvent the wheel, we use two of those actions in our first two steps to checkout the code and install Node v16 on our Ubuntu runner which we specified in the jobs block.

In the next two steps, we run two commands to install the npm dependencies and execute the test command from our package.json. This is only possible because we set up the environment in steps 1 and 2 with the pre-built actions.

These steps could easily be adapted for other environments and programming languages.

That’s it, we are done building our workflow. 🥳

Adding the workflow to the repository

To activate the workflow we just created, all we have to do is push our code with the new .github/workflows folder to our GitHub repository. The workflow will then automatically run on the specified events.

We can also go to the repository settings and set up branch protection rules for the main branch, so that pull requests into main can only be merged when the test workflow runs successfully.

If we now create a new PR into main, we can see that GitHub automatically runs our test workflow and that merging is not possible before the checks pass.

Here is a PR where the tests pass and merging is allowed.

To see an example of a failing pipeline, have a look at this PR.

(Note: You need to be logged in to a GitHub account to inspect those PRs.)

Next Steps

We have seen how to set up automated testing with just a few lines of code using GitHub Actions, pretty cool. 😎

Of course, our example was very basic and we barely scratched the surface of what GitHub Actions is capable of.

Therefore, here are some tips and resources for digging deeper:

• peruse the documentation
• extend our example project with some more workflows, e.g. for linting and formatting the code
• add GitHub Actions to more complex projects

Thanks for reading and have fun building things! 🛠️