CI/CD Pipeline for NodeJS Lambdas on AWS using Jest, Serverless Framework, Github, and TravisCI

I wanted to start a serverless project, and one of the primary headaches aside from architecting the workflow and deciding which lambda functions need to be created is a simple and yet effective CICD workflow.

Typical AWS’s suggestion is to use a combination of CodeCommit, CodeBuld, CodeDeploy, and glue everything together in CodePipeline. But I already have GitHub for other projects, so setting up CodeCommit will be unnecessary. The same applies to CodeBuild whose functions I currently perform with TravisCI, where I run both integration and feature specs.

This will be a NodeJS project, and I am interested in fully testing my functions at the Build phase using Jest. I also know I can adapt TravisCI to perform CodeDeploy’s functions in deploying code changes to production. But I still want my serverless function run on AWS Lambda. So here is my plan;

I use Github for my code repository, and Serverless Framework for creating and deploying my functions to AWS while running the Build phase on Travis. The build phase on Travis will run Jest tests on the functions, if all tests pass, it deploys the code on AWS Lambda using the serverless CLI depending on the working branch. I will create two environments, one for staging and the other for production. The staging branch deploys to my staging env, while the master branch deploys to production. Other branches will only run Jest tests on the lambda code and will not deploy.

All code used on this hands-on is available here on Github. Alright then, here we go.

Setting up the SLS functions and Github Repo

Create a new repository for our project on Github. I call mine lambda-cicd . I will clone the repository to my local workspace with:

$ git clone [email protected]:<github username>/<project name>.git

Mine is 

$ git clone [email protected]:lekansogunle/lambda-cicd.git 

Next, create two serverless AWS functions. The concept is to be able to have multiple functions in the same project and each deploying as need be to AWS Lambda. I call my two functions init-user and create-post . Hypothetically, we can say the first function is to create a new user, while the second is used to create blog posts for our user. We will only stick to the boilerplate code mostly and not implement details of these functions. Install serverless CLI with:

$ npm install -g serverless

I will create my two functions with the SLS CLI command for creating AWS Lambda services.

$ sls create --template aws-nodejs --path myService

Replacing my myService with init-user and create-post for my two functions.

$ sls create --template aws-nodejs --path init-user

$ sls create --template aws-nodejs --path create-post

Test My Services with JEST

Now that we have all the functions, I will add the Jest testing framework to the project and perform very simple testing of my functions. At the root of the project directory do:

$ yarn add --dev jest

This adds a package.json at the root directory and installs Jest. For Jest to pick up our tests, we need to add a __test__ directory in both directories holding each service. And this test folder will hold test files ending in .test.js . For consistency, since our services are named handler.js by default from SLS, I will name the tests also handler.test.js .

Next, I will go into the handler functions created for both services and change the message text to something different. For init user, I addHello from init user as a return message. My test will assert that the function returns this exact same output. Definitely, it gets far more complicated than this when the functions are fully implemented. My project file structure looks like this so far:

├── create-post
│   ├── __test__
│   │   └── handler.test.js
│   ├── handler.js
│   └── serverless.yml
├── init-user
│   ├── __test__
│   │   └── handler.test.js
│   ├── handler.js
│   └── serverless.yml
├── package.json
└── yarn.lock

Before writing the tests, now will be a good time to make a commit. Add a 

.gitignore

node_modules

node_modules

$ yarn run jest

Your test suite must contain at least one test.

I ended up having my create-post lambda function and it’s jest test look like;

'use strict';

module.exports.createPost = event => {
  return {
    statusCode: 200,
    body: JSON.stringify(
      {
        message: 'Hello From Create Post!',
        input: event,
      },
      null,
      2
    ),
  };
};

const handler = require('../handler');

test('correctly create post', () => {
  expect(handler.createPost({foo: 'bar'})).toStrictEqual({
    statusCode: 200,
    body: JSON.stringify(
      {
        message: 'Hello From Create Post!',
        input: {foo: 'bar'},
      },
      null,
      2
    ),
  });
});

Notice I changed the function name from

hello

createPost

async 

{foo: 'bar'}

strictEqual

Setup two AWS IAM account

For the next step, we will have to create two AWS IAM accounts. One for the staging environment and the other for our production environment. Here is a guide to creating IAM users. You will have to namespace the accounts with the environment they belong to (production and staging) respectively so as to clearly identify the accounts in the future. For example, production-user and staging-user. Also, follow the guide to make sure you give correct access rights. As you proceed, carefully note both Access Key Id and Secret Access Key for each user. Save these credentials in a secure place to be used in the coming steps.

Configure TravisCI

Before we configure TravisCI, we will commit everything we have and create a new branch. Call it staging and push this branch to Github. This branch will be used to create a staging deployment to AWS from TravisCI.

If you do not already have an account, create an account with TravisCI and install the Github App Integration into your Github account. Then visit https://travis-ci.com/account/repositories. Here, we are able to connect our GitHub repositories to Travis for running continuous integration jobs. Then click on settings as shown from below

The most important section here is the environment variables section. There you will notice we can set

name

value

branch

For example, staging

AWS_ACCESS_KEY_ID

AWS_ACCESS_KEY_ID

staging

We do this for all four credentials. SLS CLI will automatically pick up these variables as the setup credentials when present in the job machine. Then we add another variable,

STAGE_NAME

staging

production

Next, we create a 

.travis.yml

.travis.yml

bin/deploy

.travis.yml

bin/deploy

language: node_js
node_js:
  - "10"
cache:
  directories:
    - node_modules
    - ${SERVICE_PATH}/node_modules
install:
    - yarn global add serverless
    - travis_retry yarn install
    - cd ${SERVICE_PATH}
    - travis_retry yarn install
    - cd -
script:
    - yarn run jest
jobs:
  include:
    -
      name: "Deploy User Init API"
      env:
        - SERVICE_PATH="init-user"
    -
      name: "Deploy Create Post API"
      env:
        - SERVICE_PATH="create-post"
deploy:
  provider: script
  script: bash bin/deploy
  on:
    all_branches: true
    condition: $TRAVIS_BRANCH =~ ^(master|staging)$

#!/usr/bin/env bash

cd ${SERVICE_PATH}
serverless deploy -s ${STAGE_NAME}
cd -

Next, make sure our deploy script is executable with the following

$ chmod u+x bin/deploy

Most important parts of our 

.travis.yml

script

Otherwise, the build will be deploy after the test passes. Here, we include two sections under

job

SERVICE_PATH

deploy

script

$TRAVIS_BRANCH =~ ^(master|staging)$

The deploy script changes directory to the

SERVICE_PATH

Configure Serverless

Next, we will change some things in the

serverless.yml 

custom:
  stage: ${opt:stage, self:provider.stage}

This is to tell serverless to take in the stage option as specified when running the CLI command and then set this stage when deploying. If not present, it uses the default provided under

provider.stage

dev

Next, you will want to confirm the runtime value under the provider to be

nodejs10.x

handler.hello

handler.initUser

handler.createPost

serverless.yml

serverless.yml

service: init-user
frameworkVersion: '2'
provider:
  name: aws
  runtime: nodejs12.x
functions:
  hello:
    handler: handler.initUser

Finally, Launch Lambdas 🚀

Commit what we have and push to the staging branch. You will notice a new build started on Travis with two jobs. The jobs should pass, then I will create a pull request and merge into the master branch to deploy our production lambdas. Wait for the master branch to be green and deploy the production lambdas.

And on AWS, you will be able to see both your staging and production lambda functions deployed.

All the code for this project is available on GitHub here and TravisCI builds are available here. If you have come this far, I hope this follow-along has helped you smoothly create a lambda CICD workflow. Let me know how the process works for you!

Also published at https://medium.com/@OlalekanSogunle/create-a-ci-cd-pipeline-for-nodejs-lambdas-on-aws-using-jest-serverless-framework-github-and-d4c68dc77793