When you have a Monolith problem, smile, your business is doing very well. Software products traverse through a growth lifecycle and monoliths are good solutions for certain stages in that lifecycle.
I have been involved in the break-up of Monoliths across many different industries. The process has been much the same in all those cases. From my experience, there is a tried and trusted approach.
Monolith is an application built from a simple codebase that scales vertically. They form as part of the standard set of commonly used architectural patterns e.g. Model-View-Controller. A large amount of software frameworks advocate following monolithic patterns. These prebuilt frameworks allow the creation of software applications, platforms or SaaS, really quickly, often with canned solutions for following best practices such as the twelve factor app standard.
You quickly end up with a minimum viable product to prove out your concept with real end users in a matter of weeks rather than years. This is important because most software applications never see successful adoption by end users, so you do not want to expend too much upfront effort on non-viable products.
Monoliths are a really good way of starting projects. They have a tightly coupled architecture, how tightly coupled depends on team size and experience. Monolithic products successfully scale to be really large by increasing hardware and by evolving their design into N-Tier and then onwards, often becoming a subsystems architecture model.
You can find yourself in an organisation where you are part of a 20-50 people department taking care of just a monolithic single tier or a sub system and experiencing the monolithic pain of being too tightly coupled.
Tight coupling is expressed across both software and non software (the people, processes and hardware). For the vast majority of successful applications, the natural stage of the product lifecycle is to begin with a tightly coupled software architecture. This is down to the size of the team, the tooling available and the startup stage.
The following is a common product startup pattern that produces tight coupling.
- Team is small in membership
- Source code bunched into a small number of source repos, often with relative path dependencies
- A small number of binary artifacts make up the product
- Internal communications between processes uses low scalable options e.g. JMS, sockets etc...
- A lack of strictly defined interfaces
- The build system reflects the source code and binaries
1. A small number of long running concurrent build and test jobs
2. Limited usage of versioning
3. Libraries are built as part of the application code - The test system reflects the mantra of "everything tested together is deployed together"
- Deployment is manual, or semi automated
- Operational support is small number of people with high internal product knowledge
Signs you need to Smash a Monolith
Increasing headcount as the company grows decreases productivity, as all the engineers struggle to make code changes in reasonable turnaround time.
The underlying tooling often is struggling with longer code reviews, build, test and deployment cycles. The build pipeline is always broken, the developers are idle waiting for the machines to complete. The testing environments swing between being queued up or empty.
The deployment infrastructure itself needs to be scaled, the manual addition of machines is no longer vertically scaling due to a combination of lack of memory, cpu, network and disk storage. Deployments are frequently unsuccessful and operations are fixing forward to get systems back up and running.
The performance degrades with the number of users, as underlying resources are maxed out.
Customer and engineer satisfaction levels are nose diving.
How to Tackle Monolith Problems
A combination of the following is a recipe for success. Vision, sponsorship, buy-in, communication and strong execution.
Craft the end vision and sell it to all stakeholders. Articulate the solutions and the problems that your monolith is experiencing to the VP of engineering or equivalent to get buy-in and sponsorship. Build a vision of the state of the evolved product amongst the teams, explaining the benefits of the new architecture and the negative business impacts of waiting. It's important that the business leaders and everyone involved see the urgency to take immediate action.
Successful transformation is going to need strong prioritisation from the top, and a group of committed stakeholders to form the project leadership team.
The end vision should incorporate achieving the following
- Greater customer scalability
- Increasing engineers increases features delivered
- Efficient delivery pipeline
- Less downtime
- Increased customer and engineer satisfaction levels.
Develop a communication strategy for the migration project to the impacted stakeholders. Different styles and channels of communication are required for the leadership and the implementing teams. Ensure to include celebratory milestones to motivate with short term wins. Regularly update stakeholders on the projects progress to maintain their motivation and involve them in the milestone achievement celebrations.
Comprehensive communication is achieved both broad and deep by following the communication strategy. Hold all-hands information sessions and action all managers to cascade information down to their direct reports through their 1-2-1s/team meetings and answer any questions.
Envision your communication as a champagne fountain spreading commitment throughout the organisation. It takes time to turn the titanic of engineering opinion to align everyone on the same vision. Start by educating everyone impacted on the steps and timelines involved, include estimates for milestone deliveries on the project roadmap, that is shared freely and frequently. Send out progress update reports on a suitable cadence.
There are a number of execution strategies to change to a more distributed architecture. Identify the parts of the software product that need to change to achieve your goals, and which parts of the product delivery mechanism need to change to allow you to change those parts by acting as change levers. (sometimes it means adding in some missing parts/layers to the product architecture or changing the delivery pipeline tooling).
Software Product
Decide on how to logically split the monolith, think at least three years ahead to where your business needs to be. It will take more than a year to move to a new architecture for a big monolith.
There is a lot of pre work that you need to get in place before you start. Begin by figuring out the obvious ways to split the monolith into big chunks, keeping in mind your end product vision (solving the problems you are experiencing, scaling, developer productivity, deployment to production, complexity of codebase etc…).
Use the areas with scaling issues and those you know need rewrites as key focal points. Consider what problems the new architecture will need to solve. Look at competitor product architectures for potential reference and see if there is a next evolutionary technology you can adopt, to leap ahead (ML, AI, event sourcing/queues etc...) .
Three transformation approaches
Phased Rewrite
Rewrite your product while in motion. This is a common approach and there are some techniques to make this achievable. The problem with it is having the determination and stakeholder backing to see it through to completion. There is a risk of ending it early when most of the business goals have been achieved but there is still pain for some of the engineering teams.
Shims is a useful technique that can allow you to abstract one part of your product from another and eventually remove large sections completely while maintaining product functionality. I have been involved in this a number of times, from a large scale platform replacement of multimedia functionality to the smaller scale replacement of one database with another.
Strangulation pattern
Create a new product skeleton from scratch incorporating all you have learned from your existing product into your new architectural design. Migrate across the core functionality and features and reverse proxy to a running instance of your existing product for any missing features while you rewrite them into the new product. This gives you the time to implement the rewrite, while taking some of the the load off the existing product.
Prioritise the rewrite of features that are struggling to scale in the existing product, to allow you to continue to meet end user demand. This can feel like a race to keep your head above water, but eventually you will reach the land safely. Work with your sales team to set targets for customer onboarding to buy the time you need.
Big Bang Replacement
Complete rewrite with a new architecture, achieved by hiring a new team seeded with some seasoned engineers from the original product. Then swap in when ready. It's often the right decision if upper leadership decide to buy out a competitor with a better architecture and migrate customers over. This has a high chance of success and can be cheaper. This is not popular from engineering viewpoint.
SOA and micro-services are common distributed architectures to move towards. You need to be well read on their design patterns to migrate to using them. Modern distributed architectures contain a range of new technologies designed to achieve their goals (Containers, clusters, queues, clouds, service mesh, clusters etc...) and this will be an exciting undertaking.
Stay focused to reach your end goal. Avoid the cliff-edges of paralysis by analysis, using every bell and whistle that a new technology is offering, selecting technologies that haven't reached maturity or writing a custom solution of your own, unless its really needed and provides long term competitor advantage.
Conclusion
When you find your monolithic product has become a problem it's because your product has achieved business success. It's that successful usage that is causing you to reach a tipping point where keeping pace with end user stated and implied demands is painful due to the tight coupling that is required as part of early product lifecycle.
You need to find a way to release the pressure valve on delivery, battle the tight coupling that thwarts your best efforts. Use the process outlined above to form a team to evolve your monolith and achieve scalability and engineering productivity for continued growth and success.