How our production deployments went from 🙀 to a piece of 🍰 This blog post is part of a series where I share our migration from monolithical applications (each with their own source repository) deployed on AWS to a distributed services architecture (with all source code hosted in a monorepo) deployed on Google Cloud Platform. Part 1 (this post): “A monorepo, GitHub Flow and automation FTW” Part 2: “ One vs. many — Why we moved from multiple git repos to a monorepo and how we set it up ” Part 3: “ A (mostly) automated release process ” Part 4: “ Our approach to software development consistency ” Part 5 (coming soon): “Debug microservices locally” I think the simplest way to reduce risk in a new project is to stick to what you know. Sometimes though, it’s better to step outside your comfort zone in order to make progress. Background (a 3-tier architecture) Earlier in 2017 when I joined a new startup to lead the engineering team, I stuck to what I knew. Quickly, it became apparent that a bit of discomfort and exploration is what we needed. First though, a note on what we do: We build chatbots for market researchers to engage with their communities. The high-level requirements were: an admin web interface to author, deploy and monitor chatbots a backend system to deal with the business logic, validations and to process messages sent to the chatbots a (relational) database to persist all data This screamed “ ” all over, so a 3-tier architecture we built. 3-tier architecture How do you deploy all that code? : You stick to what you know. In my case, I previously used the . Answer Gitflow workflow Gitflow Workflow (Source: ) Atlassian Every time a feature gets merged into the branch, the branch gets deployed to a staging environment.Periodically (more on that in a moment), a branch gets created off of the branch, reviewed and merged into . As soon as that happens, gets deployed to the production environment. develop develop release develop master master We had multiple git repositories (frontend, backend, jobs service for long-running tasks and a SDK to talk to the backend’s REST API). The Gitflow workflow applied to all repositories. So, what does “periodically” mean? This is where the Gitflow workflow becomes problematic. At what point in time do you cut a branch? Every Monday morning at 9am? What if someone merges a feature into at 8:30am? Do you create a branch on Monday at 9am, test the release branch thoroughly and merge the branch into on Wednesday / Thursday / ???. Who is responsible / accountable for the production deployment? release develop release release master In the worst case, creating the branch, testing it and merging to is a tedious process. Before you know it, it’s been a month or more since you last released to production. 🙀 release master We tried all sorts of approaches, experimented with more / less automation, etc. More modularity, all code in a monorepo As our struggle with the above approach increased, we gathered in front of a whiteboard and redefined how we want to deploy to production. The main goal was to deploy to production much more frequently. In a best case scenario, we release each pull request to production as soon as it is tested in a staging environment. : Bring all code into a monorepo to better deal with dependencies. ( ) Step One This is worth its own blog post. : Get rid of the branch and only have a and branches. Wait a minute… That sounds familiar! Of course, it’s the 💡. Step Two develop master feature GitHub Flow : Automate, automate, automate! ( ) Step Three Worth another blog post… Lots of whiteboarding, experimenting, and many failed deployments later, we now have a monorepo, follow the GitHub workflow and deploy to production multiple times per day. Release Process with a monorepo and the GitHub workflow The green rectangles are fully automated, the blue rectangles require manual approval in order for the workflow to continue. Benefits With the above approach, we have a number of benefits: Deployments to production happen up to a few times per day. Each release is a fraction in size of what releases previously used to be. The responsibility to deploy code to production is shared among all team members. In our case, the reviewer of a pull request deploys to staging and production. With the GitHub flow approach, we have less branches to deal with and overall less process to get code released. In a , I will share the CircleCI configuration and a few other tools that help us automate most of the release process. follow-up post
