Scaling Microservices via Fork-ing

Building Natively Scalable Microservices

Image from LONGLONGLIFE.org

Introduction

Almost anyone familiar with microservices architecture, has heard about scaling. Scaling is the process which enables load balancing by running multiple instances of a service (container).

In any of the existing orchestration platforms, scaling can be done either automatically or manually. But, both approaches require some sort of administration deciding when a microservice needs to scale. This article introduces an autonomous scaling method driven by OS system calls without the need for external interference .

Motivation

Autonomy is one of the main principles of microservices architecture, and is essential for building agile ecosystems. In an agile ecosystem, along with agility in development, microservices require the authority to immediately react to operation conditions—high workload is one of them. When a microservice is subjected to high load, it’s more convenient and faster to reduce its workload by replicating itself.

This is somewhat similar to how body cells behave. When a cell grows, on some point, it may pass through a division phase which splits the cell to two smaller cells.

Microservices are often allowed to use a small portion of system resources. Defining the growth as the consumed amount of resources related to the total available resources, when the growth reaches a certain limit, it’s time for the microservice to replicate.

Replication via Fork-ing

Also, with this method, for sake of high availability, there’s no need for recreating the microservice on failure, as its place will be taken by new generations automatically. However, a minimum amount of replicas are required to prevent a sudden death caused by an early failure.

Implementation

Microservices are often implemented as single OS processes. The replication of a process in the OS is best known as fork-ing. The fork system call creates a child process as an exact copy of the calling process. Below is an example implementation of the replication in C using fork function:

In the above code, fork creates a new child process and returns in both the parent and its child, determined by the result of the function. Later, in the child process, execv function call is necessary to detach and hide the child process from its parent. Otherwise, when the child dies, it turns to a zombie process that won’t be cleaned up until the parent whether waits or dies.

Drawbacks

Note that the replication method explained in this article, does not remove the need for orchestration. Orchestration platforms are still required to seed the microservices, i.e., first instantiation, and controlling the system resource utilization by pushing limits on the allowed replica counts.

On the other hand, not all programming languages can interact with the OS system calls and some other may prevent them for sake of security. However, there are ways to get around this, e.g. creating new processes by executing the program directly or using a helper service.

Also obviously, fork-ing is only available within a container, and cannot instantiate new containers. This is in contrast with how current orchestration platforms scale the microservices. However, clustering can be still supported by seeding microservices on new cluster nodes.

Read More

This is the idea adopted for implementing an innovative microservices architecture, called Organic Service Architecture. You can read more about the origins of this idea and the architecture here.