Deep Dive Into Creating Node Project With Clean Architecture

(Photo by Bench Accounting on Unsplash)

How to get separation of concerns in your node app

Your architectures should tell readers about the system, not about the frameworks you used in your system — Robert C. Martin

Recently we had to build a new application in our company. After conducting business and technical design (which is out of the scope of this article), we decided that the application should be a single-page application that works with rest API. The technology stack we choose was:

Client: React
Server (API + persistent): node, elasticsearch

Coming from an object-oriented languages background, it was natural that we wanted to keep all our SOLID principles in the new and shiny node API. Like any other architecture, we had to make different trade-offs in the implementation. 

We had to be careful not to over-engineer or over-abstract our layers, but still keep it as flexible as needed.

In recent years, we have implemented clean architecture by Robert C. Martin (Uncle Bob) on our API projects. This architecture attempts to integrate some of the leading modern architecture like Hexagonal Architecture , Onion Architecture , Screaming Architecture into one main architecture. It aims to achieve good separation of concerns.

Like most architectures, it also aims to make the application more flexible to inevitable changes in client requirements (which always happens).

Node Clean Architecture Diagram

Node Clean Architecture Diagram

This diagram is taken from the official article by Robert C. Martin.

I recommend reading his article before diving into the node implementation. This is the best source knowledge about this architecture.

A few words about this diagram and how to read it(don’t worry if you don’t understand it yet, we will deep dive on each layer in this article):

1. Layers 

Each ring represents an isolated layer in the application.

2. Dependency

 The dependency direction is from the outside in. Meaning that the Entities layer is independent and the Frameworks layer (web, ui etc.) depends on all the other layers. Entities — contains all the business entities that construct our application.

3. Use Cases 

This is where we centralize our logic. Each use case orchestrates all of the logic for a specific business use case( for example adding new customer to the system).

4. Controllers and Presenters

 Our controller, presenters, and gateways are intermediate layers. You can think of them as an entry and exit gates to the use cases.

5. Frameworks 

This layer has all the specific implementations. The database, the web frameworks, error handling frameworks etc. Robert C. Martin describes this layer : this layer is where all the details go. The Web is a detail. The database is a detail. We keep these things on the outside where they can do little harm.”

At this point you are probably saying to yourself “database is in the outer layer, a database is a detail???” Database is supposed to be my core layer.

I love this architecture because it has a smart motivation behind it:

Instead of focusing on frameworks and tools this architecture focuses on the business logic of the application. It is framework independent (as much as it can:)).

This means it doesn’t matter which database, development frameworks, UI, external services you are using, the entities and the business logic of the application will always stay the same. We can change all of the above without changing our logic.

This is what makes it so easy to test applications that is built over this architecture. Don’t worry if you don’t understand this yet, we will explore it step-by-step.

In this article, we will slowly unpack the different layers of the architecture through the example of a sample app.

Like any other architecture, there are many different approaches to implement it, and each approach has its own consideration and trade-offs. In this article I will give my interpretation of how to implement this architecture on node. I will try to explain the different implementation considerations along the way.

Let’s take a closer look at the sample application.

Sample Application

Our sample app is a student registration application. The app holds a list of students, courses, and enrollments. Our backend application is a simple node API that supports all the application use-cases.

In this article, we will implement the backend API layer-by-layer. You can find all the code in the github repo. The articles contains fractions of the code, but the best approach (to my opinion) is to explore the code while reading the articles.

Entities And Use Cases

The software in this layer contains application specific business rules. It encapsulates and implements all of the use cases of the system. These use cases orchestrate the flow of data to and from the entities, and direct those entities to use their enterprise wide business rules to achieve the goals of the use case - Robert C. Martin

In the heart of the application, we have two layers:

Entities Layer - contains all the business entities that construct our application

Use Cases Layer — contains all the business scenarios that our application support.

We will walk throughout the architecture from the inside out, or the opposite direction from the dependency rule.

Inside we have independent core layers. These layers contain business and logic rules. Frameworks are rare creatures in these areas; these layers are supposed to change, mostly due to changes in the business rules.

As we go to the outer layers, we will find more frameworks and more code that changes over time due to reasons of technology or efficiency.

Entities are an independent layer and the use cases depend only on them.

In our sample application, you can find all the entities under the “src/entities” folder and all the use cases under “src/application/use_cases” folder.

Entities

The business entities in our app are:

Student:

• Id
• FirstName
• LastName
• FullName
• Enrollments

Course

• Id
• Name

Enrollment

• Course
• Grade

/Entities

module.exports = class Student {
  constructor(id = null, firstName, lastName, email, Enrollments) {
      this.id = id;
      this.firstName = firstName;
      this.lastName = lastName;
      this.email = email;
      this.Enrollments = Enrollments;
  }
};
 
module.exports = class Course {
  constructor(id = null, Name) {
      this.id = id;
      this.Name = Name;
  }
};
 
 
module.exports = class Enrollment {
  constructor(Course, Grade) {
      this.Course = Course;
      this.Grade = Grade;
  }
};

This layer is independent, which means that you will not see any

“require (‘…’)”

Use-Cases

This is where we centralize our logic. Each use case orchestrates all of the logic for a specific business use case. Our application API needs to support these use-cases:

1. Get a list of all students
2. Get single student details and enrollments
3. Add new student
4. Add enrollment to a student

Let’s examine the “add new student” use case. The use case main responsibilities are:

1. Business rules validation
2. Check that the student doesn’t exist in the DB
3. Create a new student object
4. Add the new student to the DB
5. Update the university CRM system

In order to apply validation rules on the data, you can use libraries like Joi for object schema validation . With Joi, you can describe the schema of your entities and apply validation rules before insertion. For a cleaner example (frameworks aren’t supposed to leak to the entities layer!), I have decided to keep this layer clean from external frameworks.

By looking at the use case responsibilities, we can see that the use case has two dependencies:

• Database services — the use case needs to persist the student details and check that he doesn’t exist in the system. This functionality can be implemented as a class that call SQL or MongoDB for example
• CRM services — the use case needs to notify the university CRM application about the new student. This functionality can be implemented as a class that calls Zoho or vCita CRM for example

One option is to require concrete implementations of the database and CRM services in the use case( call directly to sql sdk for example ). This option will make our database and CRM services concrete implementations tightly coupled to the use cases.

Any change in the database/CRM services (like SDK changes) will lead to changes in our use case. this option will break our clean architecture assumptions that use cases are independent and that frameworks (like database and external services) are invisible to the inner layers (like use cases).

Our use cases only knows about the entities. also testing the use cases will become harder.

OK, let’s assume the use case doesn’t know anything about concrete databases like SQL or MongoDB. He still needs to interact with them in order to perform his tasks (like persisting a student in the database), but how on earth can he do that if he doesn’t know them?

The solution is to build Gateways between the use case and the external world.

Here is were abstraction comes to the rescue. Instead of creating dependencies on a specific database or specific CRM system, we are creating dependencies on abstraction.

But what are abstractions, anyway?

Abstraction is the way of creating blueprints of a service without implementing them. In order to create abstraction, we need to do several things:

First, each use case class needs to define his dependencies as parameters in his constructor. You can see for example that the

“AddStudent”

application/use_cases/AddStudent. js

module.exports = (StudentRepository, CrmServices) => {}

Second, we need to define what we expect from each of these services. In other words, we need to define what they need to do. For example the student repository needs to have an “add student” function. At the end of this process, we would be able to define a contract.

This is a contract between the use case and the frameworks. The frameworks are the concrete database and CRM services.

What are those contracts?
Basically, the contracts are the function signatures of the desired service. For example, the CRM service needs to provide a “notify” function that gets a “student” object as a parameter and returns a promise with a Boolean value. The student repository contract tells us that the repository has to contain a get, add, update, delete functions.

In languages like C# and Java, we have interfaces and abstract classes that help us implement this kind of contract, but in js we don’t have them. We will overcome this by creating a “base class” that contains specifications without implementation.

JavaScript, unlike C#, will not enforce these contracts (meaning it won’t throw an error when the contract is not fully implemented in the concrete class). Instead, the contract will be well-documented, and the base class (you can look at it as an abstract class) will serve as a fallback if one of the methods is not implemented in the concrete class.

All the contracts are located in the application layer under contracts

application /contracts/DatabaseServices.js

class DatabaseServices {
 
  constructor() {
      this.studentRepository = null;
  }
 
  initDatabase() {
      return new Promise((resolve, reject) => {
          reject(new Error('not implemented'));
      });
  }
};

application/contracts/StudentRepository.js

module.exports = class StudentRepository {
    constructor() { }

    add(studentInstance) {
        return new Promise((resolve, reject) => {
            reject(new Error('not implemented'));
        });
    }

    update(studentInstance) {
        return new Promise((resolve, reject) => {
            reject(new Error('not implemented'));
        });
    }

    delete(studentInstance) {
        return new Promise((resolve, reject) => {
            reject(new Error('not implemented'));
        });
    }

    getById(StudentId) {
        return new Promise((resolve, reject) => {
            reject(new Error('not implemented'));
        });
    }

    getByEmail(StudentId) {
        return new Promise((resolve, reject) => {
            reject(new Error('not implemented'));
        });
    }

    getAll() {
        return new Promise((resolve, reject) => {
            reject(new Error('not implemented'));
        });
    }

    addEnrollment(studentInstance, enrollment) {
        return new Promise((resolve, reject) => {
            reject(new Error('not implemented'));
        });
    }
};

application/contracts/CrmServices.js

module.exports = class CrmServices {
 
  notify(studentDetails) {
      return new Promise((resolve, reject) => {
          reject(new Error('not implemented'));
      });
  }
};

The overriding rule that makes this architecture work is The Dependency Rule. This rule says that source code dependencies can only point inwards. Nothing in an inner circle can know anything at all about something in an outer circle. In particular, the name of something declared in an outer circle must not be mentioned by the code in the an inner circle. That includes, functions, classes. Variables, or any other named software entity — Robert C. Martin

In the diagram above, we can see the inversion of control, the use cases depend on abstraction that they define(orange arrows to gateways contracts). The use case doesn’t depend on the other layers except the entities. The outside world needs to follow these abstractions in order to talk to the use cases (they need to follow the contracts).

No dependency means that changes in the outside world won’t affect the use cases and the entities. The dependency is inverted from the flow.

Controllers and Presenters

The software in this layer is a set of adapters that convert data from the format most convenient for the use cases and entities, to the format most convenient for some external agency such as the Database or the Web. It is this layer, for example, that will wholly contain the MVC architecture of a GUI. The Presenters, Views, and Controllers all belong in here- Robert C. Martin

In the previous section, we talked about our core business layers and how they are dependent only on the abstractions that they define. Now we are going to talk about adapters, so you won’t see any business logic or frameworks here.

Controllers and Presenters diagram

Our controller, presenters, and gateways are intermediate layers. You can think of them as an adapter that glues our use cases to the outside world and back the other way.

Flow of control

Who is the outside world?

If you come from the MVC, MVVM, MVP world, you have probably heard about controllers. In classic MVC/MVP, the job of the controller is to respond to user input, validate it, do some business logic stuff, and typically mutate the state of the application.

The presenter, on the other hand, receives data from some kind of repository and formats the data for the view layer.

Controllers

In clean architecture the controller job is:

1. Receive the user input
2. Validate user input- sanitisation
3. Convert the user input into a model that the use case expects.- For example do dates format and string to integer conversion
4. .Call the use case and pass him the new model.

The controller is an adapter and we don’t want any business logic here, only data formatting logic.

Presenter

The presenter will get data from the application repository and then build a ViewModel.

Main responsibilities include:

1. Format string and dates
2. Add presentation data like flags
3. Prepare the data to be displayed in the UI

In our node implementation, we will implement the controller and the presenter together, just as we do on our MVC projects.

You can find the controllers code under the

“

src/controller

”

Controllers/students/StudentController.js

const AddStudent = require('../../application/use_cases/AddStudent');
const GetAllStudents = require('../../application/use_cases/GetAllStudents');
const GetStudent = require('../../application/use_cases/GetStudent');
const AddEnrollment = require('../../application/use_cases/AddEnrollment');

module.exports = (dependecies) => {

    const { studentRepository } = dependecies.DatabaseService;
    const { CrmServices } = dependecies;

    const addNewStudent = (req, res, next) => {
        // init use case
        const AddStudentCommand = AddStudent(studentRepository, CrmServices);
        // extract student properties
        const { firstName, lastName, email } = req.body;
        // here you can add data formatting logic
        // call use case
        AddStudentCommand.Execute(firstName, lastName, email).then((response) => {
            // here you can add data formatting logic
            res.json(response);
        }, (err) => {
            next(err);
        });
    };   
};

Few words about the implementation:

1. Dependencies are injected to the controller. It extracts what it needs and passes them down to the use case. Later we will talk about dependency injection
2. AddStudent is a factory function that returns an

   “AddStudent”

   use case object.
3. We extract the student properties from the request body object. Here we can add formatting logic, like date format and string manipulation.
4. Call the use case with the extracted properties and act with the promise result
5. After the result comes back from the use case we can adapt the result to the view. We can manipulate the model so it would be best fitted to the view logic.

Controllers and Presenters dependency diagram

The controller acts as a mediator between the outside world and the use case layer. It depends on the use cases, but they don’t depend on him

Frameworks

Node Clean Architecture — Frameworks

This layer is where all the details go. The Web is a detail. The database is a detail. We keep these things on the outside where they can do little harm- Robert C. Martin

In the previous section, we talked about the adapters layer and how they act as the entry and exit gates to our use cases. Now we are going to talk about the frameworks layer, this layer includes all our specific implementations, such as the database, the web frameworks, error handling, etc.

In our sample project, the frameworks are implemented as:

1. The web application framework is implemented by Express
2. Database services are implemented as a simple in-memory database
3. Crm services are a simple mock service

Student repository implementation using in-memory db

 Frameworks/persistence/InMemory/InMemoryStudentRepository.js

const StudentRepository = require('../../../application/contracts/StudentRepository');

module.exports = class InMemoryStudentRepository extends StudentRepository {

    constructor() {
        super();
        this.students = [];
        this.currentId = 1;
    }

    add(studentInstance) {
        return new Promise((resolve, reject) => {
            try {
                this.currentId = this.currentId + 1;
                studentInstance.id = this.currentId;
                this.students.push(studentInstance);
                resolve(studentInstance);
            } catch (error) {
                reject(new Error('Error Occurred'));
            }
        });
    }

    update(studentInstance) {
        return new Promise((resolve, reject) => {
            try {
                const student = this.students.find(x => x.id === studentInstance.id);
                if (student) {
                    Object.assign(student, { studentInstance });
                }
                resolve(student);
            } catch (error) {
                reject(new Error('Error Occurred'));
            }
        });
    }

    delete(studentId) {
        return new Promise((resolve, reject) => {
            try {
                const studentIndex = this.students.findIndex(x => x.id === studentId);
                if (studentIndex !== -1) {
                    this.students.splice(studentIndex, 1);
                }
                resolve(true);
            } catch (error) {
                reject(new Error('Error Occurred'));
            }
        });
    }

    getById(studentId) {
        return new Promise((resolve, reject) => {
            try {
                const id = parseInt(studentId);
                const student = this.students.find(x => x.id === id);
                resolve(student);
            } catch (err) {
                reject(new Error('Error Occurred'));
            }
        });
    }

    getByEmail(studentEmail) {
        return new Promise((resolve, reject) => {
            try {
                const student = this.students.find(x => x.email === studentEmail);
                resolve(student);
            } catch (err) {
                reject(new Error('Error Occurred'));
            }
        });
    }

    getAll() {
        return new Promise((resolve, reject) => {
            resolve(this.students);
        });
    }    
};

Dependency injection

As we discussed before, our use cases depend on contracts instead of implementation. These contracts needs to be satisfied via dependency injection at runtime.

If you are not familiar with the concepts of dependency injection, I encourage you to take a look at two nice videos on the Fun Fun Function blog (I love this guy :)) that explains the topic perfectly:

1. Dependency Injection basics
2. Inversion of Control

There are several libraries that provide support for dependency injection in node, but for learning purposes, I have decided to implement it without using any external library. For a larger project, it is recommended to consider using an external library for DI.

In our sample project, we have a factory function that returns an object with all the necessary dependencies of the project.

Config/projectDependencies.js

const InMemoryDatabaseServices = require('../frameworks/persistance/InMemory/InMemoryDatabaseServices');
const UniversityCrmServices = require('../frameworks/externalServices/UniversityCrmServices');

module.exports = (() => {
    return {
        DatabaseService: new InMemoryDatabaseServices(),
        CrmServices: new UniversityCrmServices()
    };
})();

This is the only file that contains information about concrete implementation. These dependencies are injected through the different layers. The framework layer is the only one that controls what will be the real implementation of all the different contracts.

Our web layer is implemented by an express server with several routing.

Now if we want to replace our in-memory database services with MongoDB for example, we need to:

1. Create a new Database Services class with the MongoDB implementation.
2. Create a new Student Repository class with the MongoDB implementation.
3. Change the

   Config/projectDependencies.js

   :

module.exports = (() => {
  return {
      DatabaseService: new MongoDBDatabaseServices(),
      CrmServices: new UniversityCrmServices()
  };
})();

Summary

In this articles, we demonstrated how to build a robust structure layer-by-layer that decouples our core business logic from frameworks. The use cases can be accessed by other clients and we are not bound to a specific web server.

We can easily replace our database with MongoDB/Elasticsearch or a move to a new CRM system, all without touching our logic. We can also react to SDK changes of one of our frameworks by touching only the framework layer. Tests are also made easy thanks to the loosely coupled architecture of the layers.

In complex projects, it is hard and sometimes tedious to keep all the layers clean and tidy. It is always about trade-offs in architecture, and every now and then we need to compromise and break our boundaries in order to get another benefit.

I believe that if we strive to keep these rules, we will get great benefits in the future.

Originally published at http://roystack.home.blog on October 22, 2019.