Continuous integration in projects using monorepo

At the beginning of every project, we have to commit to some important decisions. Among the correct decisions we made at I can cite the use of Go language, the adoption of and and our choice to store the code in a at Github. In this post, I will write about the latter, and how we solved a common challenge that the brings. Code:Nation Clean Architecture JAMStack monorepo monorepo architecture With the increasing complexity of modern projects, with and distinct user interfaces consuming resources, teams need to decide between splitting the code into multiple repositories or using a approach. Some companies like Google and DigitalOcean adopted the , as we can see in the following posts: microservices monorepo monorepo Why Google Stores Billions of Lines of Code in a Single Repository Cthulhu: Organizing Go Code in a Scalable Repo In my humble opinion, the main reasons to use a are the simplicity of repository management and the reuse of code across the team. monorepo But one of the challenges that this choice brings to the project is the potential complexity of automated build and deploy. Considering that all the code of a complex project is stored in the same repository, a change in a single file could trigger the build that can last minutes (or maybe hours) to be done. And this can be a real pain to the team, slowing the day to day productivity. In the DigitalOcean’s post that I cited before, they developed an internal solution, called (Go Test Auto), that is not open source. To solve this problem we created a similar solution but using Shell Script. gta Currently, this is our project’s directory structure: api = API and documentation chatbots = telegram, facebook and slack chatbots cli = codenation cli, used by developers to run the challenges cmd = utils and fixtures core = Go core packages, used by all the project docs = source code of internal docs (hosted at Github Pages) frontend = Vue.js project and templates used by Sam infra = configuration files used by staging and production servers lambda = lambda functions research = Python notebooks and other research assets sam = cli tool used by us to generate pages, include challenges and other admin tasks scripts = shell scrips used by CI/CD and other admin tasks web = ReactJS project (Signin, Signup, Forgot password) - IN PROCESS OF DEPRECATION workers = workers that consume SQS queues .drone.yml = CI/CD configuration file .goreleaser.yml = Goreleaser configuration file. Used to deploy the codenation-cli to Github, Homebrew docker-compose.yml = Docker configuration used by local and staging environments Gopkg.* = Go dependencies configuration files Makefile = build and admin tasks We are using as our CI/CD solution, that can we consider another good choice that we made. Our build pipeline can be seen in the image below: Drone.io And this is a sample of our config file: .drone.yml As you can see, the step executes the script , that is: golang-build-api drone_go_build_api.sh In the variable we store the list of directories that need to be monitored to our pipeline decide if the build needs to be run. This decision is made by the script : watch shouldIBuild.sh In this script, the first thing being tested is the variable , that defines if the current execution is a deploy or a build. If so, the step should run. Otherwise, the script will check if one of the directories listed in the “ ” variable has been altered by this commit. If so, the step should run. The code of is: DRONE_DEPLOY_TO watch detectChangedFolders.sh The same configuration exists in all our scripts used by Drone. That way, a change in the frontend won’t trigger the build of the API or the chatbots part of the code. Using this approach we decrease our build time from more than five minutes to a few seconds, depending on the change being commited in the repository. I believe that this approach can be used with other tools different than Drone and I hope this can help more teams to stick to the monorepo decision with confidence.