The problem Microservices were very popular a couple of years ago. Now we are all running towards monorepos! With microservices setting up Continuous integration was pretty straightforward and when they run they did relatively fast, unless you had a massive application. While monorepos are easy to set up using tools like the problem shows as soon as you try to deploy you application and set up a CI/CD pipeline. If you merge a few bigger apps suddenly it takes ages (we are talking about billable ages) for the pipeline to complete. Turborepo How to avoid it? CircleCI dynamic config to the rescue Consider the below monorepo. It consists of a design system and two applications. All applications have linting, types tests, unit tests, cypress and they even use loki visual regression tests! So it’s just slow to run everything if you made a tiny change to app2 for example! .
├── .circleci
│   ├── config.yml
│   └── continue_config.yml
├── design-system
│   ├── more_directories
├── app1
│   ├── more_directories
├── app2
│   ├── more_directories Note that we have two config files. The first one is super easy, it just checks which paths changed using a build in . config.yml Orb I added a few comments so it’s hopefully easy to read. version: 2.1

# this allows you to use CircleCI's dynamic configuration feature
setup: true

# the path-filtering orb is required to continue a pipeline based on
# the path of an updated fileset
orbs:
  path-filtering: circleci/path-filtering@0.1.1

workflows:
  # the always-run workflow is always triggered, regardless of the pipeline parameters.
  always-run:
    jobs:
      - path-filtering/filter:
          name: check-updated-files

          # Test which path is updated and set the parameter for continue_config
            design-system/.* run-design-system-job true
            app1/.* run-app1-job true
            app2/.* run-app2-job true

          # Compare changes of the branch with main branch
          base-revision: main

          # this is the path of the configuration we should trigger once
          # path filtering and pipeline parameter value updates are
          # complete. In this case, we are using the parent dynamic
          # configuration itself.

          config-path: .circleci/continue_config.yml All the fun starts in the below file, which will dynamically run only the jobs for the affected files. If you are using turborepo remote cache you might want to consider a single step build, which takes seconds if you have builds cached. version: 2.1

orbs:
  maven: circleci/maven@1.2.0

# the default pipeline parameters, which will be updated according to
# the results of the path-filtering orb
parameters:
  run-design-system-job:
    type: boolean
    default: false
  run-app1-job:
    type: boolean
    default: false
  run-app2-job:
    type: boolean
    default: false

# here we specify our workflows, most of which are conditionally
# executed based upon pipeline parameter values. Each workflow calls a
# specific job defined above, in the jobs section.

# jobs are omited for this purpose

workflows:
  # when pipeline parameter, run-design-system-job is true, the
  # jobs is triggered.
  run-design-system-job:
    when: << pipeline.parameters.run-design-system-job >>
    jobs:
      - install-dependencies
      - build
      - unit_tests
      - visuals

  run-app1-job:
    when: << pipeline.parameters.run-app1-job >>
    jobs:
      - install-dependencies
      - build
      - unit_tests

  run-app2-job:
    when: << pipeline.parameters.run-app2-job >>
    jobs:
      - install-dependencies
      - build
      - unit_tests

  run-integration-tests:
    when:
      or: [<< pipeline.parameters.run-app1-job >>, << pipeline.parameters.run-design-system-job >>]
    jobs:
      - cypress_integrations So we sped up our pipeline by not running everything every time! We just run cypress tests every time on the run-integration-tests workflow. When a change to the design system or app1 (which uses design-system) is made to make sure both play nicely with each other as you might have broken something accidentally changing a component. To speed the last workflow and cypress we could but that is a separate topic :) attach the workspace

The writer is smart, but don't just like, take their word for it. #DoYourOwnResearch

LOKI

Super

Continuous Integration for a Monorepo on CircleCI Example

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

How React Testing Library Can Improve Your Mental Health [Part 1]

10 Top Industries To Be Transformed By Artificial Intelligence, Soon

10 Essential Mobile App UI Design Principles for Building Outstanding Apps

11 AI & ML App Ideas For Startups and SME’s In 2019

11 Best Low-Code And No-Code Platforms in 2021

11 Tips to Save Time on App Development & Launch Processes

How React Testing Library Can Improve Your Mental Health [Part 1]

10 Top Industries To Be Transformed By Artificial Intelligence, Soon

10 Essential Mobile App UI Design Principles for Building Outstanding Apps

11 AI & ML App Ideas For Startups and SME’s In 2019

11 Best Low-Code And No-Code Platforms in 2021

11 Tips to Save Time on App Development & Launch Processes

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps