Deep Dive Into Creating Node Project With Clean Architecture

(Photo by on ) Bench Accounting Unsplash How to get separation of concerns in your node app Your architectures should tell readers about the system, not about the frameworks you used in your system — Robert C. Martin Recently we had to build a new application in our company. After conducting business and technical design (which is out of the scope of this article), we decided that the application should be a single-page application that works with rest API. The technology stack we choose was: Client: React Server (API + persistent): node, elasticsearch Coming from an object-oriented languages background, it was natural that we wanted to keep all our principles in the new and shiny node API. Like any other architecture, we had to make different trade-offs in the implementation. SOLID We had to be careful not to over-engineer or over-abstract our layers, but still keep it as flexible as needed. In recent years, we have implemented by Robert C. Martin (Uncle Bob) on our API projects. This architecture attempts to integrate some of the leading modern architecture like , , into one main architecture. It aims to achieve good separation of concerns. clean architecture Hexagonal Architecture Onion Architecture Screaming Architecture Like most architectures, it also aims to make the application more flexible to inevitable changes in client requirements (which always happens). Node Clean Architecture Diagram Node Clean Architecture Diagram This diagram is taken from the by Robert C. Martin. official article I recommend reading his article before diving into the node implementation. This is the best source knowledge about this architecture. A few words about this diagram and how to read it(don’t worry if you don’t understand it yet, we will deep dive on each layer in this article): 1. Layers Each ring represents an isolated layer in the application. 2. Dependency The dependency direction is from the outside in. Meaning that the Entities layer is independent and the Frameworks layer (web, ui etc.) depends on all the other layers. Entities — contains all the business entities that construct our application. 3. Use Cases This is where we centralize our logic. Each use case orchestrates all of the logic for a specific business use case( for example adding new customer to the system). 4. Controllers and Presenters Our controller, presenters, and gateways are intermediate layers. You can think of them as an entry and exit gates to the use cases. 5. Frameworks This layer has all the specific implementations. The database, the web frameworks, error handling frameworks etc. Robert C. Martin describes this layer : this layer is where all the details go. The Web is a detail. The database is a detail. We keep these things on the outside where they can do little harm.” At this point you are probably saying to yourself “database is in the outer layer, a database is a detail???” Database is supposed to be my core layer. I love this architecture because it has a smart motivation behind it: Instead of focusing on frameworks and tools this architecture focuses on the business logic of the application. It is framework independent (as much as it can:)). This means it doesn’t matter which database, development frameworks, UI, external services you are using, the entities and the business logic of the application will always stay the same. We can change all of the above without changing our logic. This is what makes it so easy to test applications that is built over this architecture. Don’t worry if you don’t understand this yet, we will explore it step-by-step. In this article, we will slowly unpack the different layers of the architecture through the example of a sample app. Like any other architecture, there are many different approaches to implement it, and each approach has its own consideration and trade-offs. In this article I will give my interpretation of how to implement this architecture on node. I will try to explain the different implementation considerations along the way. Let’s take a closer look at the sample application. Sample Application Our sample app is a student registration application. The app holds a list of students, courses, and enrollments. Our backend application is a simple node API that supports all the application use-cases. In this article, we will implement the backend API layer-by-layer. You can find all the code in the . The articles contains fractions of the code, but the best approach (to my opinion) is to explore the code while reading the articles. github repo Entities And Use Cases The software in this layer contains business rules. It encapsulates and implements all of the use cases of the system. These use cases orchestrate the flow of data to and from the entities, and direct those entities to use their business rules to achieve the goals of the use case - application specific enterprise wide Robert C. Martin In the heart of the application, we have two layers: - contains all the business entities that construct our application Entities Layer — contains all the business scenarios that our application support. Use Cases Layer We will walk throughout the architecture from the inside out, or the opposite direction from the dependency rule. Inside we have independent core layers. These layers contain business and logic rules. Frameworks are rare creatures in these areas; these layers are supposed to change, mostly due to changes in the business rules. As we go to the outer layers, we will find more frameworks and more code that changes over time due to reasons of technology or efficiency. Entities are an independent layer and the use cases depend only on them. In our sample application, you can find all the entities under the “ ” folder and all the use cases under “ ” folder. src/entities src/application/use_cases Entities The business entities in our app are: : Student Id FirstName LastName FullName Enrollments Course Id Name Enrollment Course Grade /Entities .exports = { (id = null, firstName, lastName, email, Enrollments) { .id = id; .firstName = firstName; .lastName = lastName; .email = email; .Enrollments = Enrollments; } }; .exports = { (id = null, Name) { .id = id; .Name = Name; } }; .exports = { (Course, Grade) { .Course = Course; .Grade = Grade; } }; module class Student constructor this this this this this module class Course constructor this this module class Enrollment constructor this this This layer is independent, which means that you will not see any in the entities js files. This layer wouldn’t be affected by external changes like routing or controllers, and you can persist these entities with any database (SQL, NoSQL). “require (‘…’)” Use-Cases This is where we centralize our logic. Each use case orchestrates all of the logic for a specific business use case. Our application API needs to support these use-cases: Get a list of all students Get single student details and enrollments Add new student Add enrollment to a student Let’s examine the “add new student” use case. The use case main responsibilities are: Business rules validation Check that the student doesn’t exist in the DB Create a new student object Add the new student to the DB Update the university CRM system In order to apply validation rules on the data, you can use libraries like Joi for object schema validation . With Joi, you can describe the schema of your entities and apply validation rules before insertion. For a cleaner example (frameworks aren’t supposed to leak to the entities layer!), I have decided to keep this layer clean from external frameworks. By looking at the use case responsibilities, we can see that the use case has two dependencies: — the use case needs to persist the student details and check that he doesn’t exist in the system. This functionality can be implemented as a class that call SQL or MongoDB for example Database services — the use case needs to notify the university CRM application about the new student. This functionality can be implemented as a class that calls Zoho or vCita CRM for example CRM services One option is to require concrete implementations of the database and CRM services in the use case( call directly to sql sdk for example ). This option will make our database and CRM services concrete implementations tightly coupled to the use cases. Any change in the database/CRM services (like SDK changes) will lead to changes in our use case. this option will break our clean architecture assumptions that use cases are independent and that frameworks (like database and external services) are invisible to the inner layers (like use cases). Our use cases only knows about the entities. also testing the use cases will become harder. OK, let’s assume the use case doesn’t know anything about concrete databases like SQL or MongoDB. He still needs to interact with them in order to perform his tasks (like persisting a student in the database), but how on earth can he do that if he doesn’t know them? The solution is to build between the use case and the external world. Gateways Here is were abstraction comes to the rescue. Instead of creating dependencies on a specific database or specific CRM system, we are creating dependencies on abstraction. But what are abstractions, anyway? Abstraction is the way of creating blueprints of a service without implementing them. In order to create abstraction, we need to do several things: First, each use case class needs to define his dependencies as parameters in his constructor. You can see for example that the use case expects two parameters “AddStudent” application/use_cases/AddStudent. js .exports = {} module ( ) => StudentRepository, CrmServices Second, we need to define what we expect from each of these services. In other words, we need to define what they need to do. For example the student repository needs to have an “add student” function. At the end of this process, we would be able to define a contract. This is a contract between the use case and the frameworks. The frameworks are the concrete database and CRM services. Basically, the contracts are the function signatures of the desired service. For example, the CRM service needs to provide a “notify” function that gets a “student” object as a parameter and returns a promise with a Boolean value. The student repository contract tells us that the repository has to contain a get, add, update, delete functions. What are those contracts? In languages like C# and Java, we have interfaces and abstract classes that help us implement this kind of contract, but in js we don’t have them. We will overcome this by creating a “base class” that contains specifications without implementation. JavaScript, unlike C#, will not enforce these contracts (meaning it won’t throw an error when the contract is not fully implemented in the concrete class). Instead, the contract will be well-documented, and the base class (you can look at it as an abstract class) will serve as a fallback if one of the methods is not implemented in the concrete class. All the contracts are located in the application layer under contracts application /contracts/DatabaseServices.js { () { .studentRepository = ; } initDatabase() { ( { reject( ( )); }); } }; class DatabaseServices constructor this null return new Promise ( ) => resolve, reject new Error 'not implemented' application/contracts/StudentRepository.js .exports = { () { } add(studentInstance) { ( { reject( ( )); }); } update(studentInstance) { ( { reject( ( )); }); } (studentInstance) { ( { reject( ( )); }); } getById(StudentId) { ( { reject( ( )); }); } getByEmail(StudentId) { ( { reject( ( )); }); } getAll() { ( { reject( ( )); }); } addEnrollment(studentInstance, enrollment) { ( { reject( ( )); }); } }; module class StudentRepository constructor return new Promise ( ) => resolve, reject new Error 'not implemented' return new Promise ( ) => resolve, reject new Error 'not implemented' delete return new Promise ( ) => resolve, reject new Error 'not implemented' return new Promise ( ) => resolve, reject new Error 'not implemented' return new Promise ( ) => resolve, reject new Error 'not implemented' return new Promise ( ) => resolve, reject new Error 'not implemented' return new Promise ( ) => resolve, reject new Error 'not implemented' application/contracts/CrmServices.js .exports = { notify(studentDetails) { ( { reject( ( )); }); } }; module class CrmServices return new Promise ( ) => resolve, reject new Error 'not implemented' The overriding rule that makes this architecture work is The Dependency Rule. This rule says that source code dependencies can only point inwards. Nothing in an inner circle can know anything at all about something in an outer circle. In particular, the name of something declared in an outer circle must not be mentioned by the code in the an inner circle. That includes, functions, classes. Variables, or any other named software entity — Robert C. Martin In the diagram above, we can see the inversion of control, the use cases depend on abstraction that they define(orange arrows to gateways contracts). The use case doesn’t depend on the other layers except the entities. The outside world needs to follow these abstractions in order to talk to the use cases (they need to follow the contracts). No dependency means that changes in the outside world won’t affect the use cases and the entities. The dependency is inverted from the flow. Controllers and Presenters The software in this layer is a set of adapters that convert data from the format most convenient for the use cases and entities, to the format most convenient for some external agency such as the Database or the Web. It is this layer, for example, that will wholly contain the MVC architecture of a GUI. The Presenters, Views, and Controllers all belong in here- Robert C. Martin In the previous section, we talked about our core business layers and how they are dependent only on the abstractions that they define. Now we are going to talk about adapters, so you won’t see any business logic or frameworks here. Controllers and Presenters diagram Our controller, presenters, and gateways are intermediate layers. You can think of them as an adapter that glues our use cases to the outside world and back the other way. Flow of control Who is the outside world? If you come from the MVC, MVVM, MVP world, you have probably heard about controllers. In classic MVC/MVP, the job of the controller is to respond to user input, validate it, do some business logic stuff, and typically mutate the state of the application. The presenter, on the other hand, receives data from some kind of repository and formats the data for the view layer. Controllers In clean architecture the controller job is: Receive the user input Validate user input- sanitisation Convert the user input into a model that the use case expects.- For example do dates format and string to integer conversion .Call the use case and pass him the new model. The controller is an adapter and we don’t want any business logic here, only data formatting logic. Presenter The presenter will get data from the application repository and then build a ViewModel. Main responsibilities include: Format string and dates Add presentation data like flags Prepare the data to be displayed in the UI In our node implementation, we will implement the controller and the presenter together, just as we do on our MVC projects. You can find the controllers code under the folder in the project “ src/controller ” Controllers/students/StudentController.js AddStudent = ( ); GetAllStudents = ( ); GetStudent = ( ); AddEnrollment = ( ); .exports = { { studentRepository } = dependecies.DatabaseService; { CrmServices } = dependecies; addNewStudent = { AddStudentCommand = AddStudent(studentRepository, CrmServices); { firstName, lastName, email } = req.body; AddStudentCommand.Execute(firstName, lastName, email).then( { res.json(response); }, (err) => { next(err); }); }; }; const require '../../application/use_cases/AddStudent' const require '../../application/use_cases/GetAllStudents' const require '../../application/use_cases/GetStudent' const require '../../application/use_cases/AddEnrollment' module ( ) => dependecies const const const ( ) => req, res, next // init use case const // extract student properties const // here you can add data formatting logic // call use case ( ) => response // here you can add data formatting logic Few words about the implementation: Dependencies are injected to the controller. It extracts what it needs and passes them down to the use case. Later we will talk about dependency injection AddStudent is a that returns an use case object. factory function “AddStudent” We extract the student properties from the request body object. Here we can add formatting logic, like date format and string manipulation. Call the use case with the extracted properties and act with the promise result After the result comes back from the use case we can adapt the result to the view. We can manipulate the model so it would be best fitted to the view logic. Controllers and Presenters dependency diagram The controller acts as a mediator between the outside world and the use case layer. It depends on the use cases, but they don’t depend on him Frameworks Node Clean Architecture — Frameworks This layer is where all the details go. The Web is a detail. The database is a detail. We keep these things on the outside where they can do little harm- Robert C. Martin In the previous section, we talked about the adapters layer and how they act as the entry and exit gates to our use cases. Now we are going to talk about the frameworks layer, this layer includes all our specific implementations, such as the database, the web frameworks, error handling, etc. In our , the frameworks are implemented as: sample project The web application framework is implemented by Express Database services are implemented as a simple in-memory database Crm services are a simple mock service Student repository implementation using in-memory db Frameworks/persistence/InMemory/InMemoryStudentRepository.js StudentRepository = ( ); .exports = { () { (); .students = []; .currentId = ; } add(studentInstance) { ( { { .currentId = .currentId + ; studentInstance.id = .currentId; .students.push(studentInstance); resolve(studentInstance); } (error) { reject( ( )); } }); } update(studentInstance) { ( { { student = .students.find( x.id === studentInstance.id); (student) { .assign(student, { studentInstance }); } resolve(student); } (error) { reject( ( )); } }); } (studentId) { ( { { studentIndex = .students.findIndex( x.id === studentId); (studentIndex !== ) { .students.splice(studentIndex, ); } resolve( ); } (error) { reject( ( )); } }); } getById(studentId) { ( { { id = (studentId); student = .students.find( x.id === id); resolve(student); } (err) { reject( ( )); } }); } getByEmail(studentEmail) { ( { { student = .students.find( x.email === studentEmail); resolve(student); } (err) { reject( ( )); } }); } getAll() { ( { resolve( .students); }); } }; const require '../../../application/contracts/StudentRepository' module class InMemoryStudentRepository extends StudentRepository constructor super this this 1 return new Promise ( ) => resolve, reject try this this 1 this this catch new Error 'Error Occurred' return new Promise ( ) => resolve, reject try const this => x if Object catch new Error 'Error Occurred' delete return new Promise ( ) => resolve, reject try const this => x if -1 this 1 true catch new Error 'Error Occurred' return new Promise ( ) => resolve, reject try const parseInt const this => x catch new Error 'Error Occurred' return new Promise ( ) => resolve, reject try const this => x catch new Error 'Error Occurred' return new Promise ( ) => resolve, reject this Dependency injection As we discussed before, our use cases depend on contracts instead of implementation. These contracts needs to be satisfied via dependency injection at runtime. If you are not familiar with the concepts of dependency injection, I encourage you to take a look at two nice videos on the Fun Fun Function blog (I love this guy :)) that explains the topic perfectly: Dependency Injection basics Inversion of Control There are several libraries that provide support for dependency injection in node, but for learning purposes, I have decided to implement it without using any external library. For a larger project, it is recommended to consider using an external library for DI. In our sample project, we have a factory function that returns an object with all the necessary dependencies of the project. Config/projectDependencies.js InMemoryDatabaseServices = ( ); UniversityCrmServices = ( ); .exports = { { : InMemoryDatabaseServices(), : UniversityCrmServices() }; })(); const require '../frameworks/persistance/InMemory/InMemoryDatabaseServices' const require '../frameworks/externalServices/UniversityCrmServices' module ( ) => ( return DatabaseService new CrmServices new This is the only file that contains information about concrete implementation. These dependencies are injected through the different layers. The framework layer is the only one that controls what will be the real implementation of all the different contracts. Our web layer is implemented by an express server with several routing. Now if we want to replace our in-memory database services with MongoDB for example, we need to: Create a new Database Services class with the MongoDB implementation. Create a new Student Repository class with the MongoDB implementation. Change the : Config/projectDependencies.js .exports = { { : MongoDBDatabaseServices(), : UniversityCrmServices() }; })(); module ( ) => ( return DatabaseService new CrmServices new Summary In this articles, we demonstrated how to build a robust structure layer-by-layer that decouples our core business logic from frameworks. The use cases can be accessed by other clients and we are not bound to a specific web server. We can easily replace our database with MongoDB/Elasticsearch or a move to a new CRM system, all without touching our logic. We can also react to SDK changes of one of our frameworks by touching only the framework layer. Tests are also made easy thanks to the loosely coupled architecture of the layers. In complex projects, it is hard and sometimes tedious to keep all the layers clean and tidy. It is always about trade-offs in architecture, and every now and then we need to compromise and break our boundaries in order to get another benefit. I believe that if we strive to keep these rules, we will get great benefits in the future. Originally published at http://roystack.home.blog on October 22, 2019.