Normally, when you start a project, you set your required dependencies up with the latest stable versions of all libraries and plugins. Then time goes by, the project grows, and new features and libraries are added. But the versions of the 3rd-party dependencies and the plugins remain the same; the team never updates them. Now, that’s all fine and dandy until… there׳s a conflict. If each team has its own repository, how can dependencies be managed and conflict frequency reduced? What is dependency conflict The dependency issue arises when libraries have same dependencies (internal / 3rd-party) with different versions, the library can only be installed with a single version, thus causing version conflict. Conflicting versions of 3rd-party libraries known as well as our own- appearing in production as ‘NoSuchMethodError’ or ‘ClassNotFoundError’. Repository Structure Some time back at Outbrain, we had made a transition from one big mono-repo to a multi-repo approach that required all teams to manage their dependencies by themselves, reducing visibility and control of the dependencies among multi-repos. Unfortunately, this gave rise to dependency conflicts caused by conflicting versions of 3rd-party libraries. (To read more about Outbrain’s approach to repository structures and version conflicts, check out “ Mono-repo vs Multi-repo vs Hybrid: What’s the Right Approach? ’’ After some deliberation on how to proceed, we decided on a hybrid approach: We have that is responsible for keeping internal shared libraries, versions of 3rd-party dependencies, and APIs among teams and that can be released and managed by a specific version. The teams, however, each use their own for a service code that uses library API’s, which are managed by a property version that was part of the mono-repo release process. one mono-repo multi-repos So, to use the latest internal API, we just need to upgrade specific property versions in the multi-repos. Can 3rd-party dependencies be upgraded just as easily? Yes! Centralized Dependency Management The mono-repos should reduce the frequency of dependency conflicts since we have one place with a repo that manages all dependencies. All the internal shared libraries are compiled and released with the same dependencies because they are all defined under the same pom.xml (the parent pom.xml). How is Conflict Frequency Reduced? The mono-repo has a module called “service-pom” with only one pom.xml file that contains the common plugin definitions and all the 3rd-party versions for all the deployable services. The service repos (multi-repos) must be inherited from the service-pom. In other words, the entire service-repo gets all the relevant properties from one place- the . This service-pom is released with a specific version as part of the mono-repo release process. centralized dependency management Pom Hierarchy in the Mono-repo This repo contains all our shared libraries. As part of this repository, we have modules with a structured pom hierarchy for pom’s that are part of the mono-repo release process. A Drill Down into the POM Hierarchy The service-pom This service-pom module contains only one pom.xml and is meant to be used as a parent pom for all service repos. In the image below, we can see how a service pom looks. <project>
  <parent>
    <groupId>com.outbrain</groupId>
    <artifactId>version-pom</artifactId>
    <relativePath>../version-pom</relativePath>
    <version>${revision}</version>
  </parent>
  <groupId>com.outbrain</groupId>
  <artifactId>service-pom</artifactId>
  <name>Common to all services</name>
  <version>${revision}</version>
  
 <build>
        <pluginManagement>
          <plugins>
            ...more...
          </plugins>
         </pluginManagement>
 </build>
</project> The version-pom This pom aggregates all the 3rd-party versions that we use. Below, we can see how the 3rd-party definitions are organized. <project>
  <groupId>com.outbrain</groupId>
  <artifactId>version-pom</artifactId>
  <version>${revision}</version>
  
  <properties>
    <revision>5.0.0</revision>
    <libs.version>${revision}</libs.version>
    <log4j2.version>2.17.0</log4j2.version>
    ...more...
    ...more...
  </properties>

    <dependencyManagement>
    <dependencies>
      
      <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>${log4j2.version}</version>
      </dependency>
     ...more...
     ...more...
    </dependencyManagement>
    </dependencies>  
</project> The module-pom This pom aggregates all the needs of mono-repo modules and is used as a . ּּּּּּּBelow is a layout of the internal-library definitions. bom pom <project>
  <parent>
    <groupId>com.outbrain</groupId>
    <artifactId>version-pom</artifactId>
    <relativePath>../version-pom</relativePath>
    <version>${revision}</version>
  </parent>
  <groupId>com.outbrain</groupId>
  <artifactId>modules-pom</artifactId>
  <version>${revision}</version>
  
 <dependencyManagement>
    <dependencies>
      
      <dependency>
        <groupId>com.outbrain</groupId>
        <artifactId>mono-repo-lib1</artifactId>
        <version>${libs.version}</version>
      </dependency>
      <dependency>
        <groupId>com.outbrain</groupId>
        <artifactId>mono-repo-lib2</artifactId>
        <version>${libs.version}</version>
      </dependency>
     ...more...
     ...more...
    </dependencyManagement>
    </dependencies>  
</project> Service-repos All service repositories' repo projects must: inherit from the service-pom, and add modules-pom as a bom dependency In the code block below, we have service-repo pom.xml definitions. <project>
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>com.outbrain</groupId>
    <artifactId>service-pom</artifactId>
    <version>2.0.0</version>
  </parent>
  <artifactId>service-repo-one</artifactId>
   ...
  <dependencies>
    <dependency>
        <groupId>com.outbrain</groupId>
        <artifactId>modules-pom</artifactId>
        <version>${bom.revision}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    .....
  </dependencies>
</project> As mentioned earlier, the service-pom that was released and managed by the version contains all the 3rd-party dependencies with the specific versions, since all our service-repos inherit from the service-pom Now, to upgrade 3rd-party dependencies, we can work in the mono-repo (simultaneously upgrading multiple dependencies, which will be released and managed by only one version as part of the mono-repo release process). Consequently, we can upgrade the service-pom across all the multiple repos at once. (as is shown in the image above). Upgrade Multiple Dependencies, by Managing One Version As mentioned earlier, the mono-repo can be released multiple times a day and contain the code changes of shared libraries and/or multiple 3rd-party upgrades, with all these changes being released into one version. So, all we need to update is the service-pom version across all our multi-repos. Manual Upgrades? Who Needs Them? We have an automatic tool called “Bumper” that, at a fixed frequency, upgrades and creates pull requests across all service repos with the latest version of the released version, and then merges them. Afterward, the services are automatically deployed with CI/CD pipelines. When we needed to tackle the for example, the centralized dependency management and the bumper tool took care of everything. Log4j Vulnerabilities, Got 200 repositories to update? Just head to the break room while the maven-centralized dependency management and bumper tool does it all for you.

Apache

Mono-repo Vs. Multi-repo Vs. Hybrid: What’s the Right Approach?

Read My Stories

Nominated for 2022 - HackerNoon Contributor of the Year - Java

Too Long; Didn't Read

Make resilience your competitive advantage

Keeping Dependencies Up To Date with Maven-Centralized Dependency Management

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Mono-repo Vs. Multi-repo Vs. Hybrid: What’s the Right Approach?

A Promising Future with More Opportunities: Decentralized Internet Interview with Kushagra

Can CeDeFi Offer the Best of Both Worlds?

Centralization vs Decentralization, Binance Hack, and the Binance Smart Chain Halt

Satoshi Nakamoto vs Centralized Blockchains: How Did We Get Here?

Mono-repo Vs. Multi-repo Vs. Hybrid: What’s the Right Approach?

A Promising Future with More Opportunities: Decentralized Internet Interview with Kushagra

Can CeDeFi Offer the Best of Both Worlds?

Centralization vs Decentralization, Binance Hack, and the Binance Smart Chain Halt

Satoshi Nakamoto vs Centralized Blockchains: How Did We Get Here?

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps