Normally, when you start a project, you set your required dependencies up with the latest stable versions of all libraries and plugins.
Then time goes by, the project grows, and new features and libraries are added. But the versions of the 3rd-party dependencies and the plugins remain the same; the team never updates them.
Now, that’s all fine and dandy until… there׳s a conflict. If each team has its own repository, how can dependencies be managed and conflict frequency reduced?
What is dependency conflict
The dependency issue arises when libraries have same dependencies (internal / 3rd-party) with different versions, the library can only be installed with a single version, thus causing version conflict.
Conflicting versions of 3rd-party libraries known as well as our own- appearing in production as ‘NoSuchMethodError’ or ‘ClassNotFoundError’.
Repository Structure
Some time back at Outbrain, we had made a transition from one big mono-repo to a multi-repo approach that required all teams to manage their dependencies by themselves, reducing visibility and control of the dependencies among multi-repos. Unfortunately, this gave rise to dependency conflicts caused by conflicting versions of 3rd-party libraries. (To read more about Outbrain’s approach to repository structures and version conflicts, check out “Mono-repo vs Multi-repo vs Hybrid: What’s the Right Approach?’’
After some deliberation on how to proceed, we decided on a hybrid approach: We have one mono-repo that is responsible for keeping internal shared libraries, versions of 3rd-party dependencies, and APIs among teams and that can be released and managed by a specific version. The teams, however, each use their own multi-repos for a service code that uses library API’s, which are managed by a property version that was part of the mono-repo release process.
So, to use the latest internal API, we just need to upgrade specific property versions in the multi-repos.
Can 3rd-party dependencies be upgraded just as easily? Yes!
Centralized Dependency Management
The mono-repos should reduce the frequency of dependency conflicts since we have one place with a repo that manages all dependencies.
How is Conflict Frequency Reduced?
All the internal shared libraries are compiled and released with the same dependencies because they are all defined under the same pom.xml (the parent pom.xml).
The mono-repo has a module called “service-pom” with only one pom.xml file that contains the common plugin definitions and all the 3rd-party versions for all the deployable services. The service repos (multi-repos) must be inherited from the service-pom. In other words, the entire service-repo gets all the relevant properties from one place- the centralized dependency management. This service-pom is released with a specific version as part of the mono-repo release process.
Pom Hierarchy in the Mono-repo
This repo contains all our shared libraries. As part of this repository, we have modules with a structured pom hierarchy for pom’s that are part of the mono-repo release process.
A Drill Down into the POM Hierarchy
The service-pom
This service-pom module contains only one pom.xml and is meant to be used as a parent pom for all service repos. In the image below, we can see how a service pom looks.
<project>
<parent>
<groupId>com.outbrain</groupId>
<artifactId>version-pom</artifactId>
<relativePath>../version-pom</relativePath>
<version>${revision}</version>
</parent>
<groupId>com.outbrain</groupId>
<artifactId>service-pom</artifactId>
<name>Common to all services</name>
<version>${revision}</version>
<build>
<pluginManagement>
<plugins>
...more...
</plugins>
</pluginManagement>
</build>
</project>
The version-pom
This pom aggregates all the 3rd-party versions that we use. Below, we can see how the 3rd-party definitions are organized.
<project>
<groupId>com.outbrain</groupId>
<artifactId>version-pom</artifactId>
<version>${revision}</version>
<properties>
<revision>5.0.0</revision>
<libs.version>${revision}</libs.version>
<log4j2.version>2.17.0</log4j2.version>
...more...
...more...
</properties>
<dependencyManagement>
<dependencies>
<!-- 3rd party artifacts definitions -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j2.version}</version>
</dependency>
...more...
...more...
</dependencyManagement>
</dependencies>
</project>
The module-pom
This pom aggregates all the needs of mono-repo modules and is used as a bom pom. ּּּּּּּBelow is a layout of the internal-library definitions.
<project>
<parent>
<groupId>com.outbrain</groupId>
<artifactId>version-pom</artifactId>
<relativePath>../version-pom</relativePath>
<version>${revision}</version>
</parent>
<groupId>com.outbrain</groupId>
<artifactId>modules-pom</artifactId>
<version>${revision}</version>
<dependencyManagement>
<dependencies>
<!-- Internal shared libs definitions -->
<dependency>
<groupId>com.outbrain</groupId>
<artifactId>mono-repo-lib1</artifactId>
<version>${libs.version}</version>
</dependency>
<dependency>
<groupId>com.outbrain</groupId>
<artifactId>mono-repo-lib2</artifactId>
<version>${libs.version}</version>
</dependency>
...more...
...more...
</dependencyManagement>
</dependencies>
</project>
Service-repos
All service repositories' repo projects must:
-
inherit from the service-pom, and
-
add modules-pom as a bom dependency In the code block below, we have service-repo pom.xml definitions.
<project>
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.outbrain</groupId>
<artifactId>service-pom</artifactId>
<version>2.0.0</version>
</parent>
<artifactId>service-repo-one</artifactId>
...
<dependencies>
<dependency>
<groupId>com.outbrain</groupId>
<artifactId>modules-pom</artifactId>
<version>${bom.revision}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
.....
</dependencies>
</project>
As mentioned earlier, the service-pom that was released and managed by the version contains all the 3rd-party dependencies with the specific versions, since all our service-repos inherit from the service-pom (as is shown in the image above). Now, to upgrade 3rd-party dependencies, we can work in the mono-repo (simultaneously upgrading multiple dependencies, which will be released and managed by only one version as part of the mono-repo release process). Consequently, we can upgrade the service-pom across all the multiple repos at once.
Upgrade Multiple Dependencies, by Managing One Version
As mentioned earlier, the mono-repo can be released multiple times a day and contain the code changes of shared libraries and/or multiple 3rd-party upgrades, with all these changes being released into one version.
So, all we need to update is the service-pom version across all our multi-repos.
Manual Upgrades? Who Needs Them?
We have an automatic tool called “Bumper” that, at a fixed frequency, upgrades and creates pull requests across all service repos with the latest version of the released version, and then merges them. Afterward, the services are automatically deployed with CI/CD pipelines. When we needed to tackle theLog4j Vulnerabilities, for example, the centralized dependency management and the bumper tool took care of everything.
Got 200 repositories to update? Just head to the break room while the maven-centralized dependency management and bumper tool does it all for you.