An Introduction to Serverless DevOps with OpenFaaS

DevOps isn’t about just doing CI/CD. But a CI/CD pipeline has an important role inside DevOps. I’ve been investing my time on recently and as I started creating multiple functions, I wanted an easy to use and accessible development and delivery flow, in other words a CI/CD pipeline. OpenFaaS One day as I was talking with (the creator of ), he asked me to put together a guide about CI/CD of OpenFaaS (using Github and Travis CI). It was a perfect timing for me because I was actually thinking about how I could integrate my CI/CD pipelines I apply with other projects to the OpenFaaS serverless one. What I came up with is shown on the following diagram: Alex OpenFaaS A high level overview is as below: Push to GitHub Pipeline startsBuilds function imagesCreates a temporary Swarm EnvironmentRuns tests for the functionsReleases the images to a registry if the tests passDeploys the functions if the tests pass Pipeline ends For those who want to just dive in, the repository is published . You can look inside the file to see the stages and commands of the pipeline. here .travis.yml I’ll explain the context step by step from here. Using Docker and Swarm in Travis CI Builds Alright, I’m not going into details of “how to use Travis CI”. There are plenty of good articles explaining this, and the official docs are well organized. The key points in using Travis for my use case are: We’re going to use Docker and Swarm We’re going to test using Node.js As written in the we’ll need to include: docs sudo: requiredservices: - docker in our in order to use docker in builds. In addition, we want to be able to use in order to deploy our OpenFaaS stack but this isn’t possible with Travis’ default docker version. Again, as the says, you’ll need to write: .travis.yml docker-compose version 3.2 docs before_install: - sudo apt-get update - sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce in order to install a newer version of docker. If you miss this part you’ll see unsupported Compose file version: 3.2 something like this in the pipeline and it’ll fail. Creating a Temporary Testing Environment with Swarm Open FaaS Temporary Testing Environment As shown in the diagram above, we need an OpenFaaS Environment in order to fully test the function (an e2e test). Creating a Swarm environment is easy, but preparing an OpenFaaS environment needs a little tuning for it to work properly. In order to prepare our testing environments we’ll need to: Fetch the OpenFaaS CLI tool ( ) faas-cli Initialize Swarm Deploy the Gateway and Function The OpenFaaS CLI tool ( ) makes it easy to use OpenFaaS and we’ll definitely need it for the pipeline. It’s as easy as calling this command: Fetch the OpenFaaS CLI tool: faas-cli curl -sSL | sudo sh https://cli.openfaas.com **Initialize Swarm:**This is straight forward. Just call the command: docker swarm init Okay, this is the tricky part. Normally, deploying OpenFaaS is . I really mean it. If you haven’t already, you should definitely have a look at Alex’s “ ”. But deploying it for a CI/CD pipeline adds a little concern which you normally don’t need. It’s that the environment needs to be before the test runs. That means the Swarm Service needs to be ready. To accomplish this, I wrote a simple shell script function: Deploy the Gateway and Function: extremely easy FaaS and Furious — 0 to Serverless in 60 seconds, anywhere ready # This function checks if the service is in Running statecheck_service_is_running() {local SERVICE_NAME=$1local STATE=$(docker service ps --format '{{json .CurrentState}}' $SERVICE_NAME)if [[ $STATE = \"Running* ]]; thenecho 1elseecho 0fi} This uses to check the state of the service. If it is it returns and other than that, it returns . In addition to this functionality, we’ll need to retry until the service is ready so I added a retry function as well (it calls internally): docker service ps Running 1 0 check_service_is_running # This function waits for the service to become available.# Retries for 10 times and 3 second interval (hard coded for now)wait_for_service_to_start() {local n=1local max=10local delay=3 local SERVICE_NAME=$1local SERVICE_IS_RUNNING=0while [ "$SERVICE_IS_RUNNING" -eq 0 ]; doif [[ $n -gt $max ]]; thenecho "ERROR: Retried $(($n-1)) times but $SERVICE_NAME didn't start. Exiting" >&2exit 1fiSERVICE_IS_RUNNING=$(check_service_is_running $SERVICE_NAME)echo "Waiting for $SERVICE_NAME to start"n=$[$n+1]sleep $delaydoneecho "$SERVICE_NAME is Running"} Now that we can ensure the gateway and functions are running as expected, we can setup the environment with the commands below: # deploy the stack to swarm./deploy_stack.sh # build the functions (assuming 4 cores)faas-cli build --parallel 4 -f stack.yml # we can't deploy unless the gateway is ready so waitwait_for_service_to_start func_gateway# and then deployfaas-cli deploy -f stack.yml # wait for functions to become ready for testingwait_for_service_to_start echo The is simply calling: deploy_stack.sh docker stack deploy func --compose-file docker-compose.yml and deploying the stack inside Swarm. I want to reuse this shell on my local machine hence it is in an external file. creates the function images and deploys the functions via gateway. With this tweak, we can ensure a fully working OpenFaaS environment (you can check the complete script inside the file). faas-cli build faas-cli deploy ci-setup.sh Testing the Function Now that we have a testing environment, testing against it is pretty straight forward. You can choose any framework of your choice, but for this article I chose Node.js and the library. The following test is a sample test for the function. Just checking if the response is the text I sent: chakram echo const chakram = require('chakram');const expect = chakram.expect; const ENDPOINT = " "; http://localhost:8080/function/echo describe("FaaS echo function", () => {it("should respond with the data you passed", () => {// Arrangeconst expected = "echo test"; // Actreturn chakram.post(ENDPOINT, expected, {json: false}).then(response => {// Assertexpect(response).to.have.status(200);expect(response.body).to.contain(expected);});});}); Don’t forget to add the following lines to in order to use Node.js and cache libraries: .travis.yml language: node_jsnode_js: "8" cache: yarn Cleaning up Swarm This is probably not necessary but I like to be symmetric so I’ve decided to clean up the Swarm environment: docker swarm leave -f Release and Deploy After the tests pass as expected, we’d like to release the function images and deploy them to other environments (dev, stage, prod, whatever). This is straight forward and it is written in the travis ci , too: docs after_success: if [ "$TRAVIS_BRANCH" == "master" ]; thendocker login -u="$DOCKER_USERNAME" -p="$DOCKER_PASSWORD";faas-cli push -f echo.ymlfi This means the images are pushed only if the and it is on the . pushes the functions to the registry. Another interesting part is the and environment variable. You can set these credentials via the travis ci management console. tests succeed master branch faas-cli push $DOCKER_USERNAME $DOCKER_PASSWORD travis ci dashboard From the Settings, you can set Environment Variables: Setting Environment Variables Hence you don’t have to reveal your credentials inside the . While the commands above aren’t deploying to any environment, it is just a matter of triggering some deploy webhook (or something similar to that). So you’ll be executing some kind of trigger command here regardless of what backend you’re using (Swarm, Kubernetes, Cattle, etc). .travis.yml Run the Pipeline! Everything is prepared! Now let’s make some edits and trigger the pipeline. You should see something similar to this: You should see the tests passing and the images getting released as well. An automated full pipeline! Wrapping Up In this post we’ve created a full CI/CD pipeline for OpenFaaS. You can replace the services to the ones you prefer, GitLab, Gogs, Jenkins, Circle CI, GoCD, Drone, etc. (you name it) and you can use them seamlessly on Cloud or On-Premise, anywhere you’d like. Needless to say but being able to completely take over control of the pipeline is a huge advantage! Still, there are some things we could probably improve including: Updating the docker every time to use the docker-compose in the beginning of the pipeline takes a bit time. Wish there was some kind of way to switch the docker version in Travis ver: 3.2 The CI downloads the every time. Perhaps we could create a container and reuse it in the pipeline taking benefits of image caching. faas-cli faas-cli If you have any thoughts, please feel free to share them with ! Would love feedback, too! me Further reading: You can find the complete repo of this article’s project here: https://github.com/kenfdev/faas-echo If you haven’t already you should definitely take a look at Alex’s “OpenFaaS: From Zero to Serverless in 60 Seconds Anywhere with Alex Ellis”: If you’re interested in OpenFaaS, please show support by giving a Star to the GitHub repo !