The rise of real-time event-driven applications has led to the development of modern technology stacks that can handle large volumes of events in real time. Real-time event-driven applications are characterized by their ability to respond immediately to events as they occur, providing users with up-to-date information and faster feedback. To build a modern stack for a real-time event-driven application, you need to consider several components and technologies that can handle the different stages of event processing, from event collection to user interface. In this article, we will discuss the various components of a modern stack in a real-time event-driven sample online application architecture that from different markets in a city to customers. provides discount information Learning objectives By the end of this article, you will learn: Understand the limitations of continuous polling data from a server. Build event-driven architecture for a discount web application. How to show data to users in real-time. Understanding the app use cases and requirements A website or mobile application that provides discount information from different markets in a city to customers can be a useful tool for shoppers who are looking to . The app can provide real-time information about discounts and deals in various markets, allowing users to quickly and easily find the best deals. save money on their purchases The below picture shows when we open the website on a mobile app: Customers can use this website to and about discounts and deals at markets that are close to their current location as soon as those markets create to the system. They can use the app to plan their shopping trips and budget their purchases based on the available discounts and deals. compare prices at different markets receive notifications real-time entries Some core technical requirements for this app may include: It should provide about discounts and deals at various markets, ensure that customers have access to the most up-to-date information, and filter out this data based on the user’s location. Maybe it also should allow users to and deals based on specific products or categories. Implement to provide personalized deals information based on the user's preferences. real-time information search for discounts user authentication Here is an important question arises: Without diving into many details in this website architecture, let’s only focus on finding a solution for the technical requirement of . how do we show real-time discount information to users (while they are using the website) as this data appears in city markets? retrieving discount data from the server in real-time Evaluate the first typical solution The first possible straightforward architecture reports discount information by fetching changes from the server based on and shows them on a frontend app. The page uses a timer to send a request to the server every three seconds to request discounts. The response returns an array of discounts, which are then displayed to the user. a timer single-page This design is known as a polling approach. The sample client might use libraries such as to compose the website UI and the HTTP client to handle requests to a backend API endpoint (The backend can be built using REST and any back-end framework such as or any low-code framework). timer-based JavaScript ReactJS Axios NodeJS The discount information is regularly updated by another integrated API that faces external calls from different markets (It can use to be notified when new market discounts get available by providing a call-back endpoint) and then it stores on the server in a relational database (MySQL, PostgreSQL). When our backend discount API is triggered by an HTTP request from the client website, it returns content from the database. a webhook endpoint Limitations of the current solution The actual issue lies in not how fast we ingest market discount data from data providers, but in how fast we deliver it to UI (Assuming that data is already stored in our database, we analyze and process this data). Let's think about some of . the weaknesses of this approach Between the discount frontend app and backend service, there are constant poll requests for changes. Because it is timer-based, the client application contacts the server whether or not changes exist to the underlying data. Once data is returned from the server, the is updated on the web page - regardless of any changes in the data. This is highly inefficient, and calls when there haven't been any updates in the database. Also, what if the called HTTP API accepts our HTTP request but , this could affect the user experience, especially when the behavior is reflected in the user interface (meaning a user has to refresh a page to get the latest changes on discount). entire list of discounts may result in empty payloads takes a long time to handle it Once you're more familiar with the first web app architecture and its limitations, it's time to introduce a new design to solve the above issues. Real-Time Event-Driven Data Exchange seems an ideal fit to achieve the above technical requirements. In an event-driven architecture, components are designed to , rather than being invoked by other components or services. Exactly what we want, instead of having to constantly poll for changes from the backend, it uses a and lets the backend server send a message or notification to all connected clients automatically. Event-Driven Architecture (EDA) react to events as they occur push-based approach Here, refers to the ability of our system to respond to events or input immediately, or within a very short period of time. In the context of the discount web app, it means processing discount events as soon as they occur, without any significant delay. Real-time The below diagram shows a new architecture of components in the discount app. The architecture shows 4 starting from how we discount data change, and them to event consumers and them on the UI. Basically, it is a reverse flow to the first solution’s design. main stages detect ingest propagate display Let’s break down each component and understand their roles in the next section. New architecture breakdown This new design adds real-time features to our discount data, reduces traffic, and makes a more efficient UI by only updating as data changes. But it leverages a few open-source technologies and tools for event streaming. The first component is a that acts as a data source, which can be PostgreSQL (Other popular options include MongoDB or MySQL). As data changes in the database, a change is detected using the Log-based ( ) capabilities of the database. It captures the change and records it in a transaction log. The captured changes are then transformed into a change event that can be consumed in real-time by downstream systems (a message broker) such as . database CDC Change Data Capture Kafka We will use the to extract these CDC logs in the form of event streams from the database to Kafka. Once our discount events come into Kafka, a such as can subscribe to this change feed from Kafka topics. Then, RisingWave reads and stores them in its local persistent storage as to achieve . In other words, it can act as a stateful stream processor that can stream into relational database tables that represent the current state. The streaming database helps us quickly build a materialized view in real time that shows all discount information based on user-specified matching criteria, or prepare statistics like the TOP 5 deals on a chosen product or find out the closest deals from results of different markets. Additionally, it gives us the ability to analyze the data by delivering them to for making better business decisions based on our web application usage. Debezium connector for Postgres streaming database RisingWave materialized views fault tolerance materialize the CDC events BI and data analytics platforms is a streaming database specialized in real-time analytics that can read directly database change events from and build a materialized view by joining multiple events together. will keep the view up to date as new events come and allow you to query using SQL to have access to the latest changes made to the data. RisingWave Postgres binlogs or Kafka topics RisingWave The streaming database writes back results to Kafka topics by using in real-time. Now, we need to add JavaScript code to consume and process discount messages received from Kafka and update the UI in real-time to display them. We can use the library, which is a popular Kafka client for Node.js to listen and consume Kafka messages. sink operation KafkaJS Now, you can use this Kafka consumer function in a React Native component to update the UI as new discount data is received. Conclusion In this post, we learned how to design architecture for a web app that provides real-time discount information from different markets. The web app can help users find the best deals and make informed shopping decisions. We evaluated two different solutions that implement time-triggered polling and use even-driven patterns. We understood that building a modern stack for a real-time event-driven application requires a range of technologies. One can use a combination of streaming processors, the streaming database, modern JavaScript frameworks, and libraries. Streaming database deals with complex queries that often change by pre-computing the result in its cache. In the second architecture, we did not introduce any additional service/microservice that implements a custom stream processing logic that would result in increased operational overhead and complexity of deployments. Related resources How to choose the right streaming database Issue SQL queries to manage your data Query Real-Time Data in Kafka Using SQL How Change Data Capture (CDC) Works with Streaming Database Recommended content Is RisingWave the Right Streaming Database? Rethinking Stream Processing and Streaming Databases Community 🙋 Join the Risingwave Community About the author Visit my personal blog: www.iambobur.com Also published . here

The writer is smart, but don't just like, take their word for it. #DoYourOwnResearch before making any investment decisions or decisions regarding you health or security. (Do not regard any of this content as professional investment advice, or health advice)

An Intro to API Gateways for ChatGPT Plugins

Join my journey to learning!

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

A Modern Stack to Build a Real-time Event-driven App

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

An Intro to API Gateways for ChatGPT Plugins

Apache Kafka: How Does It Work?

Auto-generation of Documentation for Event-driven Architecture

Building Efficient Crypto Trading Software With Python

Building Notification Systems with the End Users In Mind

An Intro to API Gateways for ChatGPT Plugins

Apache Kafka: How Does It Work?

Auto-generation of Documentation for Event-driven Architecture

Building Efficient Crypto Trading Software With Python

Building Notification Systems with the End Users In Mind

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps