This blog post is part of a series where I share our migration from monolithical applications (each with their own source repository) deployed on AWS to a distributed services architecture (with all source code hosted in a monorepo) deployed on Google Cloud Platform.
I think the simplest way to reduce risk in a new project is to stick to what you know. Sometimes though, it’s better to step outside your comfort zone in order to make progress.
Earlier in 2017 when I joined a new startup to lead the engineering team, I stuck to what I knew. Quickly, it became apparent that a bit of discomfort and exploration is what we needed. First though, a note on what we do: We build chatbots for market researchers to engage with their communities. The high-level requirements were:
This screamed “3-tier architecture” all over, so a 3-tier architecture we built.
Answer: You stick to what you know. In my case, I previously used the Gitflow workflow.
Gitflow Workflow (Source: Atlassian)
Every time a feature gets merged into the develop
branch, the develop
branch gets deployed to a staging environment.Periodically (more on that in a moment), a release
branch gets created off of the develop
branch, reviewed and merged into master
. As soon as that happens, master
gets deployed to the production environment.
We had multiple git repositories (frontend, backend, jobs service for long-running tasks and a SDK to talk to the backend’s REST API). The Gitflow workflow applied to all repositories.
This is where the Gitflow workflow becomes problematic. At what point in time do you cut a release
branch? Every Monday morning at 9am? What if someone merges a feature into develop
at 8:30am? Do you create a release
branch on Monday at 9am, test the release branch thoroughly and merge the release
branch into master
on Wednesday / Thursday / ???. Who is responsible / accountable for the production deployment?
In the worst case, creating the release
branch, testing it and merging to master
is a tedious process. Before you know it, it’s been a month or more since you last released to production. 🙀
We tried all sorts of approaches, experimented with more / less automation, etc.
As our struggle with the above approach increased, we gathered in front of a whiteboard and redefined how we want to deploy to production.
The main goal was to deploy to production much more frequently. In a best case scenario, we release each pull request to production as soon as it is tested in a staging environment.
Step One: Bring all code into a monorepo to better deal with dependencies. (This is worth its own blog post.)
Step Two: Get rid of the develop
branch and only have a master
and feature
branches. Wait a minute… That sounds familiar! Of course, it’s the GitHub Flow 💡.
Step Three: Automate, automate, automate! (Worth another blog post…)
Lots of whiteboarding, experimenting, and many failed deployments later, we now have a monorepo, follow the GitHub workflow and deploy to production multiple times per day.
Release Process with a monorepo and the GitHub workflow
The green rectangles are fully automated, the blue rectangles require manual approval in order for the workflow to continue.
With the above approach, we have a number of benefits:
In a follow-up post, I will share the CircleCI configuration and a few other tools that help us automate most of the release process.