Event-driven Microservices with Quebic

Hi Today I am going to discuss about how to develop Event-driven using Quebic . If you are new to microservices you can refer my previous which covered main concepts about microservices. And If you are new to Quebic please go through with this . microservices framework article documents Main Topics Why we need Event-driven Microservices Event Sourcing Event-driven CQRS Sample Application - Event driven coffee shop Events flow Getting started with Quebic Quebic - EventBox Implementation How to run sample application 1. Why we need Event-driven Microservices No Foreign key constrain between services Assume you have two services. One service user-service and other one is search-service that service is optimized for searching. When user-service stores new user-data it also call search-service to store related data. If someone delete user-data from user-service , The search-service don’t know about that deletion. Distributed transactions are hard When some transaction is happened through the multiple services, If some issue occur within the execution , Since transaction touched multiple places then hard to identify who is the responsible for the reproduce/rollback. High coupling between services When we want to exchange information between services we need to know about each others. Then needs to know how to reach another service and how to response it. Hard to scale In the traditional Microservices architecture all the inter-services communication are happen request-response way. If one service is slow all the caller stack of that service getting slow. That’s make hard to scale the system. Lets see how we can use Event Sourcing and Event-driven architecture concepts to minimize these challenges. 2. Event Sourcing Event Sourcing stores every state transactions as an event in a log. All the events are stored for a particular domain. That domain-events are the main key concept behind in the Event Sourcing architecture. The domain-event is a data structure that representation a some thing happen in the past. Domain-events are immutable and immortal. That’s mean never change or never remove it from Event-Store. With Event Sourcing architecture, we have an audit-trail to tell how things happen in the past. Now we know how to reproduce / rollback the state of the system. This is really help to identify issues occurs in distributed systems. 3. Event-driven The Event-driven architecture helps to remove the high coupling between services. Each services can run independently and no need to know about how to reach others. Every one listening for there own events. With Event-driven Microservices, all the operations are happened in eventually consistence way. Instead of waiting and get the response, user get tracker to track his request. But eventually he can get the correct response. Lets see how to implement Event-driven Microservices using these concepts. 4. CQRS CQRS stands for Command Query Responsibility Segregation. This pattern is used to create materialized views from streams of events. CQRS pattern is built based on Event-sourcing and Event-driven architecture concepts. According to the CQRS pattern , you can divide your application’s read and write parts into separate modules. Read part is called query model and write part is called command model. But it is not required to keep command and query models into separate projects. If you want you can keep both modules in a single project then those module are running on same process. But those module are fully independent, any time you can separate those. CQRS separates reads from writes. Then it helps to scale both parts independently. Each sides are optimized. OK I think, we have covered most of the theories. Lets jump into implementation. 5. Sample Application - Event driven coffee shop Here you can see we have three Microservices. orders-service, beans-service and barista-service. Source . location is responsible for manage orders which coming from users. orders-service is responsible for storing and validating beans before process the order. beans-service is responsible for making coffee according to the order. barista-service 6. Events flow I am not going to explain whole event flow, but I will explain some of the important points. Here first step is is listening for event. orders-service request.OrderPlaced That event is fired by API-Gateway. After is received the event, then it creates and response back that to the API-Gateway. Then user can use that to track his order eventually. orders-service reference-id reference-id reference-id Then invokes the method. That method is responsible for creating domain-event. The ( created in previous step ) is used as the of the domain-event. order-service placeOrder() orders.OrderPlaced reference-id aggregate-id 7. Getting started with Quebic Quebic is a framework for writing serverless functions to run on Kubernetes. You can write your functions in any language. Currently Quebic supports only for Java and NodeJS. Github . repo 8. Quebic - EventBox EventBox enables the Event-Sourcing capability of the Quebic. With Eventbox, you can maintain Event-store for your domains and EventBox provides way to inspect domain-events under particular domains. That makes help to get advantages of Event-Sourcing into your applications. You can access EventBox by using its REST-Interface or Dashboard UI. Create Domain Dashboard UI Go to the Event Store section and then click create button. Then it will open Domain create interface. There are three mandatory field. Domain name. Provide unique name for your domain for the identification Name : This will be the prefix value of the event types. Lets take some events types. Event Group : Eg: , orders.OrderPlaced orders.OrderBeanValidated Here all the events types are started with ‘orders’ prefix. That is the event group of that event. By mentioning event-group, This domain is going to listening only for the event types. orders.* Aggregate field name of the domain-event. You can set the identity field name of the domain object. Aggregate ID Field : REST-Interface curl -X POST http:// /api/domains-H 'content-type: application/json'-d '{"name":"orders","eventGroup":"orders","aggregateIdField":"id"}' eventbox-host:port Fetch Domain-Events Dashboard UI Go to the Event Store section and It will display created domains. Open a right-menu from one of that domains. Select Domain-Event link of the menu. Then it will be redirected to the domain-events pages. REST-Interface curl http:// /api/domains/ /domain-events eventbox-host:port 9. Implementation According our sample project we have three Microservices , and . We can divided into two modules and according to the CQRS pattern. We are going to implement each modules using Quebic functions. orders-service beans-service barista-service orders-service orders-commond-service orders-query-service Here I am going to explain some important points. You can find whole implementation to this sample application in . here orders-command-service In orders-commond-service, users are allowed only for write operations. This is a part of the it’s . function-spec file ...replicas: 1events: request.OrderPlaced orders.OrderBeanValidated orders.OrderBeanCancelled orders.CoffeeBrewStarted orders.CoffeeBrewFinished According to the events section, this function is going to listening only for , , etc.. events. request.OrderPlaced orders.OrderBeanValidated Let’s check its . It use for mapping request into particular methods. handler implementation context.eventID orders-query-service In orders-query-service, users are allowed only for read operations. When receive an event which start from ‘orders’ ( eg: ) prefix, Then the event’s payload is stored in the query data-store. For this example I just used JavaScript data structure as the query data-store. But if you want you can use some external database for query data-store. orders.OrderPlaced Let’s check its . You can see, Its call method when loading function-container. Inside that It makes a call for the EventBox to get latest event’s payloads for a particular domain. handler implementation fetchLatestPayloads() http:// /api/domains/ /domain-events/latest-payloads eventbox-host:port EventBox stores all domain events, This endpoint represent the current state of the system under particular domain. When start/re-start it invokes method. This will help to up to date query data-store with domain-event-store and it make sure all the instances of the keep same query data-store. orders-query-service fetchLatestPayloads() orders-query-service You can get the Connection URI of the EventBox using environment variable. EVENTBOX_URI Routing rules let’s take a look at . This is a sample http-request for placing new order. This will return created order with order-id. order-place routing rule /orders POST api-gateway-host:port Content-Type: "application/json"{"beanType":"bean_type_1"} let’s take a look at This is a sample http-request to fetch order details. order-fetch routing rule. /orders/{order-id} GET api-gateway-host:port After user send the http-request to API-Gateway then it fires event. The is received that event then get the relevant order details from the query data-store. request.OrderFetch orders-query-service This is the which created when placing the order. And this is the of the domain event. order-id reference-id order-id aggregate-id 10. How to run sample application This contains all instructions to run sample application. document I think you got a considerable knowledge about Event-driven Microservices and how to implement it with Quebic. Please contact me if there are any problems. I really expecting your valuable feedback. Thanks for reading. Good luck. References