12/12/2024: Top 5 stories on the HackerNoon homepage!

The HackerNoon Newsletter: What The Heck is DeltaStream? (12/12/2024)

COBOL implementation of Apache Kafka and Iceberg.

Announcing the COBOL Streamhouse

A brief run-through of DeltaStream and how it simplifies working with streaming data such as Kinesis and Apache Kafka, taking advantage of Apache Flink.

What The Heck is DeltaStream?

DeltaStream

deltastream

When we last asked, “What the heck is???”, we were looking at the __Apache Kafka__® protocol compatible data streaming platform WarpStream. With streaming data, you need to do something with it. Land it somewhere like Apache Iceberg, Snowflake, or Databricks, or query that in-flight data to enrich and/or filter it before landing it. There are some options out there for the latter, but the biggest one is undoubtedly the open-source project Apache Flink. Looking at Flink led me to DeltaStream, which is our subject today. What the heck is DeltaStream? How does it work with Flink, and how is it a companion to Kafka? What is Flink? Flink was initially accepted as an Apache project in December 2014, so it has been around for a while. The growth of stream processing has led to accelerated interest and use in the last few years. Flink is a somewhat challenging system to stand up and use internally, requiring dedicated engineering talent. Even the AWS MSF service, while easier, is still fairly complex. Utilizing Java, for example, requires you to write your Java query, generate a jar file, zip it, upload it to S3, set your permissions, and then execute it. I’m going to borrow from the Apache Flink web page here. Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments and perform computations at in-memory speed and at any scale. I don’t want to replicate more of what is on their website, so if you aren’t sure about Flink, give it a look. Using DeltaStream At its most basic, DeltaStream gives you the power of Flink without having to deal with Flink. At all. For my test, I used their Quick Start Guide for the Web UI; note that they also have a CLI, which I did not test. When you sign up, you get a free 14-day trial. A sample Kafka cluster called “trial_store” is spun up that you can use instead of your own data to try out. The demo contains a number of topics to play with. The icons under ACTIONS allow you to delete a topic or view details and content about and in the topic. Here are the details of the pageviews topic for reference, which we’ll use later. Okay, we’ve got a Kafka cluster going and topics in it. Now, let’s do something interesting. As mentioned in the intro, the most interesting thing I can do is enrich and/or filter data while it is in flight before it lands at its ultimate destination, like a database/warehouse/lake. For those purposes, we go to the Workspace. This part took a little getting used to. A database and a schema in DeltaStream are just organizational folders. You can create ‘n’ number of databases, and within a database, you can have ‘n’ number of schemas. The schemas will hold the definitional layout of your DeltaStream Objects known as STREAM, CHANGELOG, MATERIALIZED VIEW, and TABLE. A Table relates to a database table in something like PostgreSQL, and the Materialized View is a way to persist the data in a result set without putting it somewhere specifically. I’m not going to do anything with either of those in this blog; I’m going to focus on Stream and Changelog. I’m creating a Stream of the pageviews topic in the Kafka cluster in the following screenshot. I think of it as making a table definition of the topic. We’ve assigned the names of the fields and told them what topic to use and what the data format is. We don’t have to fully justify the trial_store cluster as it is set as the default in the combo box at the top. Once that command is executed, it will show up under TestDB.public. I can then query it with something like SELECT * FROM PAGEVIEWS, and I’ll start seeing the data in the result pane at the bottom. Next, I declare a changelog backed by the user's topic and ordered by UserID. A changelog is similar to a stream but enables you to interpret events in a topic as UPSERT events. Events require a primary key; DeltaStream interprets each event as an insert or update for the given primary key. In this case, the changelog reflects specific details by user, such as gender and interests. Here is where we start to have some fun. I will create a new stream that enriches the pageviews stream with data from the users_log changelog using the userid as the key value to join on. This now gives me a unique topic in the cluster with data from two different topics. From here, I can filter it using something like regionid and write the results of that query to a final destination, such as a database, warehouse, or lake. This allowed me to enrich and filter data in-flight before landing it, thus improving latency and reducing compute and storage costs. Summary So, what the heck is DeltaStream? It’s a really simple way to use Apache Flink without knowing anything about it or directly doing anything with it. You saw from my example how straightforward it was to connect to Apache Kafka and then read, join, and filter the data. Other currently supported connections are Kinesis, PostgreSQL, Snowflake, and Databricks, and I’m told that ClickHouse and Iceberg will soon be available. Ultimately, DeltaStream gives you the power of Apache Flink without having to deal with Apache Flink, and you can do it using SQL instead of Java. If you are dealing with streaming data or looking to implement it, this is definitely a very clever and convenient solution. Check out my other What the Heck is… articles at the links below: What The Heck Is DuckDB?
What the Heck Is Malloy?
What the Heck is PRQL?
What the Heck is GlareDB?
What the Heck is SeaTunnel?
What the Heck is LanceDB?
What the heck is SDF?
What the Heck is Paimon?
What the Heck is Proton?
What the Heck is PuppyGraph?
What the Heck is GPTScript?
What the Heck is WarpStream? ** When we last asked, “What the heck is???”, we were looking at the __ Apache Kafka __® protocol compatible data streaming platform WarpStream . With streaming data, you need to do something with it. Land it somewhere like Apache Iceberg , Snowflake , or Databricks , or query that in-flight data to enrich and/or filter it before landing it. There are some options out there for the latter, but the biggest one is undoubtedly the open-source project Apache Flink . Looking at Flink led me to DeltaStream , which is our subject today. What the heck is DeltaStream? How does it work with Flink, and how is it a companion to Kafka? Apache Kafka WarpStream WarpStream Apache Iceberg Apache Iceberg Snowflake Snowflake Databricks Databricks Apache Flink Apache Flink DeltaStream DeltaStream What is Flink? Flink was initially accepted as an Apache project in December 2014, so it has been around for a while. The growth of stream processing has led to accelerated interest and use in the last few years. Flink is a somewhat challenging system to stand up and use internally, requiring dedicated engineering talent. Even the AWS MSF service, while easier, is still fairly complex. Utilizing Java, for example, requires you to write your Java query, generate a jar file, zip it, upload it to S3, set your permissions, and then execute it. AWS MSF AWS MSF I’m going to borrow from the Apache Flink web page here. Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments and perform computations at in-memory speed and at any scale. I don’t want to replicate more of what is on their website, so if you aren’t sure about Flink, give it a look. Using DeltaStream At its most basic, DeltaStream gives you the power of Flink without having to deal with Flink. At all. For my test, I used their Quick Start Guide for the Web UI; note that they also have a CLI, which I did not test. Quick Start Guide Quick Start Guide When you sign up, you get a free 14-day trial. A sample Kafka cluster called “trial_store” is spun up that you can use instead of your own data to try out. The demo contains a number of topics to play with. The icons under ACTIONS allow you to delete a topic or view details and content about and in the topic. Here are the details of the pageviews topic for reference, which we’ll use later. pageviews Okay, we’ve got a Kafka cluster going and topics in it. Now, let’s do something interesting. As mentioned in the intro, the most interesting thing I can do is enrich and/or filter data while it is in flight before it lands at its ultimate destination, like a database/warehouse/lake. For those purposes, we go to the Workspace. This part took a little getting used to. A database and a schema in DeltaStream are just organizational folders. You can create ‘n’ number of databases, and within a database, you can have ‘n’ number of schemas. The schemas will hold the definitional layout of your DeltaStream Objects known as STREAM, CHANGELOG, MATERIALIZED VIEW, and TABLE. A Table relates to a database table in something like PostgreSQL, and the Materialized View is a way to persist the data in a result set without putting it somewhere specifically. I’m not going to do anything with either of those in this blog; I’m going to focus on Stream and Changelog. I’m creating a Stream of the pageviews topic in the Kafka cluster in the following screenshot. I think of it as making a table definition of the topic. We’ve assigned the names of the fields and told them what topic to use and what the data format is. We don’t have to fully justify the trial_store cluster as it is set as the default in the combo box at the top. Once that command is executed, it will show up under TestDB.public. I can then query it with something like SELECT * FROM PAGEVIEWS, and I’ll start seeing the data in the result pane at the bottom. pageviews Next, I declare a changelog backed by the user's topic and ordered by UserID. A changelog is similar to a stream but enables you to interpret events in a topic as UPSERT events. Events require a primary key; DeltaStream interprets each event as an insert or update for the given primary key. In this case, the changelog reflects specific details by user, such as gender and interests. user's Here is where we start to have some fun. I will create a new stream that enriches the pageviews stream with data from the users_log changelog using the userid as the key value to join on. This now gives me a unique topic in the cluster with data from two different topics. From here, I can filter it using something like regionid and write the results of that query to a final destination, such as a database, warehouse, or lake. This allowed me to enrich and filter data in-flight before landing it, thus improving latency and reducing compute and storage costs. pageviews users_log userid Summary So, what the heck is DeltaStream? It’s a really simple way to use Apache Flink without knowing anything about it or directly doing anything with it. You saw from my example how straightforward it was to connect to Apache Kafka and then read, join, and filter the data. Other currently supported connections are Kinesis, PostgreSQL, Snowflake, and Databricks, and I’m told that ClickHouse and Iceberg will soon be available. Ultimately, DeltaStream gives you the power of Apache Flink without having to deal with Apache Flink, and you can do it using SQL instead of Java. If you are dealing with streaming data or looking to implement it, this is definitely a very clever and convenient solution. Check out my other What the Heck is… articles at the links below: What The Heck Is DuckDB? What the Heck Is Malloy? What the Heck is PRQL? What the Heck is GlareDB? What the Heck is SeaTunnel? What the Heck is LanceDB? What the heck is SDF? What the Heck is Paimon? What the Heck is Proton? What the Heck is PuppyGraph? What the Heck is GPTScript? What the Heck is WarpStream? What The Heck Is DuckDB? What The Heck Is DuckDB? What The Heck Is DuckDB? What the Heck Is Malloy? What the Heck Is Malloy? What the Heck Is Malloy? What the Heck is PRQL? What the Heck is PRQL? What the Heck is PRQL? What the Heck is GlareDB? What the Heck is GlareDB? What the Heck is GlareDB? What the Heck is SeaTunnel? What the Heck is SeaTunnel? What the Heck is SeaTunnel? What the Heck is LanceDB? What the Heck is LanceDB? What the Heck is LanceDB? What the heck is SDF? What the heck is SDF? What the heck is SDF? What the Heck is Paimon? What the Heck is Paimon? What the Heck is Paimon? What the Heck is Proton? What the Heck is Proton? What the Heck is Proton? What the Heck is PuppyGraph? What the Heck is PuppyGraph? What the Heck is PuppyGraph? What the Heck is GPTScript? What the Heck is GPTScript? What the Heck is GPTScript? What the Heck is WarpStream? What the Heck is WarpStream? What the Heck is WarpStream? **

Co je sakra DeltaStream?

¿Qué diablos es DeltaStream?

دلتا استریم چیست؟

מה זה לעזאזל DeltaStream?

Mi a fene az a DeltaStream?

DeltaStream って一体何ですか?

អ្វី​ដែល​ The Heck ​គឺ​ជា​ DeltaStream?

O que diabos é DeltaStream?

DeltaStream යනු කුමක්ද?

Waa maxay Heck waa DeltaStream?

DeltaStream คืออะไรกันแน่?

Що таке DeltaStream?

Introduction COBOL (Common Business Oriented Language) still plays a significant role in the world's codebase, particularly in critical business and financial systems. While exact figures vary, recent estimates suggest that the amount of COBOL code still in use is substantial: There are 775-850 billion lines of COBOL code in daily use worldwide.
COBOL systems handle approximately $3 trillion worth of daily transactions.
43% of banking systems still use COBOL.
95% of ATM swipes in the US and 80% of in-person credit card transactions are processed using COBOL systems.
Approximately 70% to 80% of the world's business transactions are processed in COBOL. COBOL has been all over the news these days for various reasons. Here are some recent links for you to enjoy before we dive into the meat of what is happening. The World Depends on 60-Year-Old Code No One Knows Anymore
134k Lines Of Code Posted As Latest Effort For COBOL Support Within GCC
Unlocking new potential: Transform your Assembler programs to COBOL with AWS Mainframe Modernization
Social Security's COBOL Software Comes Under Scrutiny
Cobol knowledge crisis threatens Dutch financial systems
From COBOL And Assembly To Java—Modernizing The IRS
Banks will "maybe" be done with COBOL... in ten years While COBOL may not be as prevalent in new development, it remains a critical component of many legacy systems, particularly in banking, finance, insurance, and government industries. Despite its age, COBOL remains relevant due to its reliability, efficiency, and suitability for business-oriented tasks. To that end, DeltaStream is introducing the COBOL Streamhouse! What is the COBOL Streamhouse? For decades, COBOL has been the backbone of critical systems in industries like finance, insurance, and government. Despite COBOL’s reputation for reliability, its resistance to modernization has left many organizations grappling with a paradox: how do you preserve the value of battle-tested legacy code while unlocking the agility and scalability of today’s data-driven world? The COBOL Streamhouse project is our answer—a bold initiative to integrate the modern data stack with COBOL, starting with streaming and data lakehouse capabilities that promise to breathe new life into these stalwart systems. The purpose of COBOL Streamhouse is straightforward yet ambitious: to bring the benefits of real-time data processing and advanced analytics to applications that have, until now, been marooned in a batch-processing past. We’re not here to rip and replace—COBOL’s enduring presence is proof of its worth—but to extend its capabilities, ensuring it can thrive alongside cloud-native technologies. By meeting COBOL on its own terms, we’re crafting a bridge between yesterday’s code and tomorrow’s possibilities, empowering organizations to leverage their existing investments without the disruption of a complete rewrite. Our journey begins with two foundational phases that tackle the most pressing needs in modern data architectures: streaming and data management. The first phase introduces streaming to COBOL via an Apache Kafka (CobKa) implementation explicitly tailored for this legacy language. Kafka, renowned for its ability to handle high-throughput, real-time data streams, is a cornerstone of the modern data stack. Bringing it to COBOL means enabling these systems to process transactions, logs, and events as they happen—not just in nightly batches. Imagine a banking system that can analyze real-time payment flows or an insurance platform that adjusts risk models on the fly. Our CobKa implementation retains the language’s native strengths—reliability, precision, and compatibility—while embedding the low-latency, event-driven power of streaming. Complementing this, the second phase introduces Apache Iceberg, reimagined in COBOL as CobBerg, to create a robust data lakehouse framework. Iceberg’s appeal lies in its ability to manage massive datasets with features like schema evolution, partitioning, and ACID transactions—capabilities that are light-years ahead of the flat-file approaches common in legacy COBOL environments. By implementing Iceberg in COBOL, we’re equipping these systems to handle structured and semi-structured data at scale, all while maintaining transactional integrity. This isn’t just about storing data; it’s about making it queryable, governable, and ready for modern analytics tools, from SQL engines to machine learning frameworks. Together, Kafka and Iceberg in COBOL lay the groundwork for a data ecosystem that’s both legacy-friendly and future-ready. These initial phases are just the beginning. We’re addressing the most immediate gaps between COBOL and contemporary architectures by starting with streaming and data lakehouse functionality. The Kafka implementation unlocks real-time insights, while Iceberg provides a foundation for scalable, flexible data management. Moreover, we’re doing this without forcing developers to abandon COBOL’s syntax or paradigms—our implementations are designed to feel native, not bolted-on. This approach preserves the expertise of COBOL programmers, many of whom have spent decades mastering the language, while inviting a new generation to see its potential in a modern context. COBOL Streamhouse isn’t about rewriting history; it’s about rewriting the future of legacy systems. As we roll out these capabilities, we’re proving that modernization doesn’t have to mean migration. With streaming and data lakehouse technologies now within reach, COBOL can step confidently into the era of real-time, data-centric computing—without losing the reliability that made it indispensable in the first place. Summary If you want to learn more about COBOL in the context of a language you understand, I wrote this multi-part article for the HP Chronicle magazine in 1986. It was intended to teach C to COBOL programmers because COBOL was more prevalent than C then. Make sure to check out the COBOL Streamhouse GitHub. All the code is available to view and use. Sample source code and binaries of a producer and a consumer are included. We’re excited about this development and the opportunities it presents. Introduction Introduction COBOL (Common Business Oriented Language) still plays a significant role in the world's codebase, particularly in critical business and financial systems. While exact figures vary, recent estimates suggest that the amount of COBOL code still in use is substantial: There are 775-850 billion lines of COBOL code in daily use worldwide. COBOL systems handle approximately $3 trillion worth of daily transactions. 43% of banking systems still use COBOL. 95% of ATM swipes in the US and 80% of in-person credit card transactions are processed using COBOL systems. Approximately 70% to 80% of the world's business transactions are processed in COBOL. There are 775-850 billion lines of COBOL code in daily use worldwide. 775-850 billion lines 775-850 billion lines COBOL systems handle approximately $3 trillion worth of daily transactions. $3 trillion worth $3 trillion worth 43% of banking systems still use COBOL. 43% of banking systems 43% of banking systems 95% of ATM swipes in the US and 80% of in-person credit card transactions are processed using COBOL systems. 95% of ATM swipes 95% of ATM swipes Approximately 70% to 80% of the world's business transactions are processed in COBOL. 70% to 80% 70% to 80% COBOL has been all over the news these days for various reasons. Here are some recent links for you to enjoy before we dive into the meat of what is happening. The World Depends on 60-Year-Old Code No One Knows Anymore 134k Lines Of Code Posted As Latest Effort For COBOL Support Within GCC Unlocking new potential: Transform your Assembler programs to COBOL with AWS Mainframe Modernization Social Security's COBOL Software Comes Under Scrutiny Cobol knowledge crisis threatens Dutch financial systems From COBOL And Assembly To Java—Modernizing The IRS Banks will "maybe" be done with COBOL... in ten years The World Depends on 60-Year-Old Code No One Knows Anymore The World Depends on 60-Year-Old Code No One Knows Anymore The World Depends on 60-Year-Old Code No One Knows Anymore 134k Lines Of Code Posted As Latest Effort For COBOL Support Within GCC 134k Lines Of Code Posted As Latest Effort For COBOL Support Within GCC 134k Lines Of Code Posted As Latest Effort For COBOL Support Within GCC Unlocking new potential: Transform your Assembler programs to COBOL with AWS Mainframe Modernization Unlocking new potential: Transform your Assembler programs to COBOL with AWS Mainframe Modernization Unlocking new potential: Transform your Assembler programs to COBOL with AWS Mainframe Modernization Social Security's COBOL Software Comes Under Scrutiny Social Security's COBOL Software Comes Under Scrutiny Social Security's COBOL Software Comes Under Scrutiny Cobol knowledge crisis threatens Dutch financial systems Cobol knowledge crisis threatens Dutch financial systems Cobol knowledge crisis threatens Dutch financial systems From COBOL And Assembly To Java—Modernizing The IRS From COBOL And Assembly To Java—Modernizing The IRS From COBOL And Assembly To Java—Modernizing The IRS Banks will "maybe" be done with COBOL... in ten years Banks will "maybe" be done with COBOL... in ten years Banks will "maybe" be done with COBOL... in ten years While COBOL may not be as prevalent in new development, it remains a critical component of many legacy systems, particularly in banking, finance, insurance, and government industries. Despite its age, COBOL remains relevant due to its reliability, efficiency, and suitability for business-oriented tasks. To that end, DeltaStream is introducing the COBOL Streamhouse ! COBOL remains relevant COBOL remains relevant DeltaStream DeltaStream COBOL Streamhouse COBOL Streamhouse What is the COBOL Streamhouse? For decades, COBOL has been the backbone of critical systems in industries like finance, insurance, and government. Despite COBOL’s reputation for reliability, its resistance to modernization has left many organizations grappling with a paradox: how do you preserve the value of battle-tested legacy code while unlocking the agility and scalability of today’s data-driven world? The COBOL Streamhouse project is our answer—a bold initiative to integrate the modern data stack with COBOL, starting with streaming and data lakehouse capabilities that promise to breathe new life into these stalwart systems. The purpose of COBOL Streamhouse is straightforward yet ambitious: to bring the benefits of real-time data processing and advanced analytics to applications that have, until now, been marooned in a batch-processing past. We’re not here to rip and replace—COBOL’s enduring presence is proof of its worth—but to extend its capabilities, ensuring it can thrive alongside cloud-native technologies. By meeting COBOL on its own terms, we’re crafting a bridge between yesterday’s code and tomorrow’s possibilities, empowering organizations to leverage their existing investments without the disruption of a complete rewrite. Our journey begins with two foundational phases that tackle the most pressing needs in modern data architectures: streaming and data management. The first phase introduces streaming to COBOL via an Apache Kafka (CobKa) implementation explicitly tailored for this legacy language. Kafka, renowned for its ability to handle high-throughput, real-time data streams, is a cornerstone of the modern data stack. Bringing it to COBOL means enabling these systems to process transactions, logs, and events as they happen—not just in nightly batches. Imagine a banking system that can analyze real-time payment flows or an insurance platform that adjusts risk models on the fly. Our CobKa implementation retains the language’s native strengths—reliability, precision, and compatibility—while embedding the low-latency, event-driven power of streaming. Complementing this, the second phase introduces Apache Iceberg, reimagined in COBOL as CobBerg, to create a robust data lakehouse framework. Iceberg’s appeal lies in its ability to manage massive datasets with features like schema evolution, partitioning, and ACID transactions—capabilities that are light-years ahead of the flat-file approaches common in legacy COBOL environments. By implementing Iceberg in COBOL, we’re equipping these systems to handle structured and semi-structured data at scale, all while maintaining transactional integrity. This isn’t just about storing data; it’s about making it queryable, governable, and ready for modern analytics tools, from SQL engines to machine learning frameworks. Together, Kafka and Iceberg in COBOL lay the groundwork for a data ecosystem that’s both legacy-friendly and future-ready. These initial phases are just the beginning. We’re addressing the most immediate gaps between COBOL and contemporary architectures by starting with streaming and data lakehouse functionality. The Kafka implementation unlocks real-time insights, while Iceberg provides a foundation for scalable, flexible data management. Moreover, we’re doing this without forcing developers to abandon COBOL’s syntax or paradigms—our implementations are designed to feel native, not bolted-on. This approach preserves the expertise of COBOL programmers, many of whom have spent decades mastering the language, while inviting a new generation to see its potential in a modern context. COBOL Streamhouse isn’t about rewriting history; it’s about rewriting the future of legacy systems. As we roll out these capabilities, we’re proving that modernization doesn’t have to mean migration. With streaming and data lakehouse technologies now within reach, COBOL can step confidently into the era of real-time, data-centric computing—without losing the reliability that made it indispensable in the first place. Summary If you want to learn more about COBOL in the context of a language you understand, I wrote this multi-part article for the HP Chronicle magazine in 1986. It was intended to teach C to COBOL programmers because COBOL was more prevalent than C then. this multi-part article this multi-part article Make sure to check out the COBOL Streamhouse GitHub . All the code is available to view and use. Sample source code and binaries of a producer and a consumer are included. We’re excited about this development and the opportunities it presents. GitHub GitHub producer consumer

Aankondiging van die COBOL Streamhouse

Anuncio del COBOL Streamhouse

Lähde: COBOL Streamhouse

Descargar cancións de COBOL Streamhouse

COBOL Streamhouseの発表

ការបង្ហាញពី COBOL Streamhouse

„COBOL Streamhouse“ pristatymas

Manambara ny COBOL Streamhouse

Przedstawiamy COBOL Streamhouse

Ku dhawaaqida COBOL Streamhouse

Објављивање COBOL Streamhouse

Tungkol sa COBOL Streamhouse

T3Go is China’s first platform for smart travel based on the Internet of Vehicles. In this article, Trevor Zhang and Vino Yang from T3Go describe the evolution of their data lake architecture, built on cloud-native or open-source technologies, including Alibaba OSS, Apache Hudi, and Alluxio. Today, their data lake stores petabytes of data, supporting hundreds of pipelines and tens of thousands of tasks daily. It is essential for business units at T3Go, including Data Warehouse, Internet of Vehicles, Order Dispatching, Machine Learning, and self-service query analysis. In this blog, you will see how we slashed data ingestion time by half using Hudi and Alluxio. Furthermore, data analysts using Presto, Hudi, and Alluxio saw the queries speed up by 10 times. We built our data lake based on data orchestration for multiple stages of our data pipeline, including ingestion and analytics. T3Go data lake Overview Prior to the data lake, different business units within T3Go managed their own data processing solutions, utilizing different storage systems, ETL tools, and data processing frameworks. Data for each became siloed from every other unit, significantly increasing cost and complexity. Due to the rapid business expansion of T3Go, this inefficiency became our engineering bottleneck. We moved to a unified data lake solution based on Alibaba OSS, an object store similar to AWS S3, to provide a centralized location to store structured and unstructured data, following the design principles of ; all the applications access OSS storage as the source of truth, as opposed to different data silos. This architecture allows us to store the data as-is without having to first structure the data and run different types of analytics to guide better decisions, building dashboards and visualizations from big data processing, real-time analytics, and machine learning. Multi-cluster Shared-data Architecture Efficient Near Real-time Analytics Using Hudi Our business in smart travel drives the need to process and analyze data in a near real-time manner. With a traditional data warehouse, we faced the following challenges: High overhead when updating due to long-tail latency. High cost of order analysis due to the long window of a business session Reduced query accuracy due to late or ad-hoc updates Unreliability in data ingestion pipeline. Data lost in the distributed data pipeline cannot be reconciled. High latency to access data storage. As a result, we adopted Apache Hudi on top of OSS to address these issues. The following diagram outlines the architecture: Enable near real-time data ingestion and analysis With Hudi, our data lake supports multiple data sources, including Kafka, MySQL binlog, GIS, and other business logs in near real-time. As a result, more than 60% of the company’s data is stored in the data lake, and this proportion continues to increase. We are also able to speed up the data ingestion time down to a few minutes by introducing Apache Hudi into the data pipeline. Combined with big data interactive query and analysis frameworks, such as Presto and SparkSQL, real-time data analysis and insights are achieved. Enable Incremental processing pipeline With the help of Hudi, it is possible to provide incremental changes to the downstream derived table when the upstream table updates frequently. Even with a large number of interdependent tables, we can quickly run partial data updates. This also effectively avoids updating the full partitions of cold tables in the traditional Hive data warehouse. Accessing Data using Hudi as a unified format Traditional data warehouses often deploy Hadoop to store data and provide batch analysis. Kafka is used separately to distribute Hadoop data to other data processing frameworks, resulting in duplicated data. Hudi helps effectively solve this problem; we always use Spark pipelines to insert new updates into the Hudi tables, then incrementally read the update of Hudi tables. In other words, Hudi tables are used as the unified storage format to access data. Efficient Data Caching Using Alluxio In the early version of our data lake without Alluxio, data received from Kafka in real-time is processed by Spark and then written to OSS data lake using Hudi DeltaStreamer tasks. With this architecture, Spark often suffered high network latency when writing to OSS directly. Since all data is in OSS storage, OLAP queries on Hudi data may also be slow due to a lack of data locality. To address the latency issue, we deployed Alluxio as a data orchestration layer, co-located with computing engines such as Spark and Presto, and used Alluxio to accelerate read and write on the data lake as shown in the following diagram: Data in formats such as Hudi, Parquet, ORC, and JSON are stored mostly on OSS, consisting of 95% of the data. Computing engines such as Flink, Spark, Kylin, and Presto are deployed in isolated clusters, respectively. When each engine accesses OSS, Alluxio acts as a virtual distributed storage system to accelerate data, being co-located with each of the computing clusters. Specifically, here are a few applications leveraging Alluxio in the T3Go data lake. Data lake ingestion We mount the corresponding OSS path to the Alluxio file system and set Hudi’s parameter value to use the scheme in place of scheme. Spark pipelines with Hudi continuously ingest data to Alluxio. After data is written to Alluxio, it is asynchronously persisted from the Alluxio cache to the remote OSS every minute. These modifications allow Spark to write to a local Alluxio node instead of writing to remote OSS, significantly reducing the time for the data to be available in the data lake after ingestion. target-base-path alluxio:// oss:// Data analysis on the lake We use Presto as an ad-hoc query engine to analyze the Hudi tables in the lake, co-locating Alluxio workers on each Presto worker node. When Presto and Alluxio services are co-located and running, Alluxio caches the input data locally in the Presto worker, which greatly benefits Presto for subsequent retrievals. On a cache hit, Presto can read from the local Alluxio worker storage at memory speed without any additional data transfer over the network. Concurrent accesses across multiple storage systems In order to ensure the accuracy of training samples, our machine learning team often synchronizes desensitized data in production to an offline machine learning environment. During synchronization, the data flows across multiple file systems, from production OSS to an offline HDFS followed by another offline Machine Learning HDFS. This data migration process is not only inefficient but also error-prone for modelers because multiple different storages with varying configurations are involved. Alluxio helps in this specific scenario by mounting the destination storage systems under the same filesystem to be accessed by their corresponding logical paths in Alluxio namespace. By decoupling the physical storage, allows applications with different APIs to access and transfer data seamlessly. This data access layout also improves performance. Microbenchmark Overall, we observed the following improvements with Alluxio: It supports a hierarchical and transparent caching mechanism. It supports cache promote mode when reading. It supports asynchronous writing mode. It supports LRU recycling strategy. It has pin and TTL features After comparison and verification, we choose to use Spark SQL as the query engine. Our performance testing queries the Hudi table, comparing Alluxio + OSS together against OSS directly as well as HDFS. In the stress test shown above, after the data volume is greater than a certain magnitude (2400W), the query speed using Alluxio+OSS surpasses the HDFS query speed of the hybrid deployment. After the data volume is greater than 1E, the query speed starts to double. After reaching 6E data, it is up to 12 times higher than querying native OSS and 8 times higher than querying native HDFS. The improvement depends on the machine configuration. Based on our performance benchmarking, we found that the performance can be improved by over 10 times with the help of Alluxio. Furthermore, the larger the data scale, the more prominent the performance improvement. Next Step As T3Go’s data lake ecosystem expands, we will continue facing the critical scenario of compute and storage segregation. With T3Go’s growing data processing needs, our team plans to deploy Alluxio on a larger scale to accelerate our data lake storage. In addition to the deployment of Alluxio on the data lake computing engine, which currently is mainly SparkSQL, we plan to add a layer of Alluxio to the OLAP cluster using Apache Kylin and an ad_hoc cluster using Presto. The goal is to have Alluxio cover all computing scenarios, with Alluxio interconnected between each scene to improve the read and write efficiency of the data lake and the surrounding lake ecology. Conclusion As mentioned earlier, Hudi and Alluxio cover all scenarios of Hudi’s near real-time ingestion, near real-time analysis, incremental processing, and data distribution on DFS, among many others, and play the role of a powerful accelerator on data ingestion and data analysis on the lake. With Hudi and Alluxio together, our R&D engineers shortened the time for data ingestion into the lake by up to a factor of 2. Data analysts using Presto, Hudi, and Alluxio in conjunction to query data on the lake saw their queries speed up by 10 times faster. Furthermore, the larger the data scale, the more prominent the performance improvement. Alluxio is an important part of T3Go’s plan to become a leading enterprise data lake in China. We look forward to seeing further integration with Alluxio in T3Go’s data lake ecosystem.

Building High-Performance Data Lake Using Apache Hudi and Alluxio 

Event Time Processing with Flink and Beam - Power of Real time Analytics

How to Implement Real-Time Credit Card Fraud Detection With Apache Flink on AWS

Every application is built from the same type of components - buttons, inputs, dropdowns, etc., and modal windows are in this list. The reason is simple - modal windows allow us to communicate with a user and do not overwhelm a UI with additional information until it becomes needed. In this article, I will explore one of the approaches to managing modal windows in apps. Angular We use the primeng library for modals in the article's examples. Your case can be different. The general idea will be the same despite the libraries, but API can be different; be aware of it. Modal windows are just regular components - they have markup, style, and selector. But in contrast to regular components, they usually don't have to be shown on the page immediately. We need them only for the result of users' actions - confirm, discard, fill a form, etc. That's why we won't include them in our build - we will download them dynamically only if we require them. Let's see an example: public openModal(): void {
    const modalComponent = async (): Promise<any> => import('./path-to-your-modal');
    this.modalService.open(modalComponent());
} Here, we have the dynamic import approach. It means that our component will be loaded when we invoke the method. It's quite useful because we don't load the modal immediately in our chunk - only when the user needs it. This approach can help you to slightly reduce the bundle size, especially if your modals are full of logic and components. openModal In general, this is the working solution. But we can go further. Let's make a service that will handle the dynamic imports and return an instance of the modal: import { Injectable } from '@angular/core';
import { DialogService } from 'primeng/dynamicdialog';

@Injectable({ providedIn: 'root' })
export class CustomModalService {
    public modalWindowComponents: { [key: string]: () => Promise<any> } = {
        ConfirmModalComponent: () =>
            import('your-path/to-component').then((component) => component.ConfirmModalComponent),
    };

constructor(private modalService: DialogService) {}

    public async openDynamicDialogModal({
        componentName,
        data,
    }: {
        componentName: string;
        data?: { [key: string]: any };
    }): Promise<any> {
        const modalComponent = this.modalService.open(await this.modalWindowComponents[componentName](), { ...data });

        modalComponent.onClose.subscribe(() => {
            // some logic
        });
        return modalComponent;
    }
} Here, we can see several important things: We use an object to store modal windows imports where the key is the component name, and the value is an import function that returns Promise; Our method is taking the params object with the component's name (to take the import function from the object) and some additional data, which contains whatever you want for a component or its appearing configuration. Also, we have a subscription for close events. It is more convenient to handle window closing in one place; We return a Promise with the imported component. It allows us to handle the logic for each modal. Here is an example of the usage of this service: public async handleOpeningModalWindow(): Promise<void> {
    const dialogRef = await this.primengModalService.openDynamicDialogModal({
        componentName: 'DeleteStreamModalComponent',
        data: {
           // your params
        },
    });

    dialogRef.onClose
        .pipe(
            // your pipes
        )
        .subscribe(() => {
            // your logic
        });
} Conclusion By adopting the approach outlined above, Angular developers can achieve a leaner, more manageable modal window system. This strategy not only reduces bundle size but also offers a unified method for modal control and customization. Embracing dynamic imports and service-based architecture empowers developers to optimize user experiences while maintaining code scalability and efficiency.

How to Manage Modal Windows in Angular Apps

How to Build and Deploy a Real-time Traffic Monitoring App Using Quix's Computer Vision Template

কুইক্সের কম্পিউটার ভিশন টেমপ্লেট ব্যবহার করে কীভাবে একটি রিয়েল-টাইম ট্র্যাফিক মনিটরিং অ্যাপ তৈরি এবং স্থাপন করবেন

So erstellen und implementieren Sie eine Echtzeit-Verkehrsüberwachungs-App mithilfe der Computer Vision-Vorlage von Quix

Cómo crear e implementar una aplicación de monitoreo de tráfico en tiempo real utilizando la plantilla de visión por computadora de Quix

Comment créer et déployer une application de surveillance du trafic en temps réel à l'aide du modèle de vision par ordinateur de Quix

क्विक्स के कंप्यूटर विज़न टेम्पलेट का उपयोग करके रीयल-टाइम ट्रैफ़िक मॉनिटरिंग ऐप कैसे बनाएं और तैनात करें

Quix の Computer Vision テンプレートを使用してリアルタイム トラフィック監視アプリを構築および展開する方法

Quix의 컴퓨터 비전 템플릿을 사용하여 실시간 교통 모니터링 앱을 구축하고 배포하는 방법

Como construir e implantar um aplicativo de monitoramento de tráfego em tempo real usando o modelo de visão computacional do Quix

Как создать и развернуть приложение для мониторинга трафика в реальном времени с использованием шаблона компьютерного зрения Quix

Quix'in Görüntü İşleme Şablonunu Kullanarak Gerçek Zamanlı Trafik İzleme Uygulaması Nasıl Oluşturulur ve Dağıtılır

Cách xây dựng và triển khai ứng dụng giám sát lưu lượng truy cập thời gian thực bằng mẫu thị giác máy tính của Quix

如何使用 Quix 的计算机视觉模板构建和部署实时流量监控应用程序

Flink-Doris-Connector 1.4.0 allows users to ingest a whole database ( or ) that contains thousands of tables into , a real-time analytic database, . MySQL Oracle Apache Doris in one step With built-in Flink CDC, the Connector can directly synchronize the table schema and data from the upstream source to Apache Doris, which means users no long have to write a DataStream program or pre-create mapping tables in Doris. When a Flink job starts, the Connector automatically checks for data equivalence between the source database and Apache Doris. If the data source contains tables that do not exist in Doris, the Connector will automatically create those same tables in Doris, and utilize the side outputs of Flink to facilitate the ingestion of multiple tables at once; if there is a schema change in the source, it will automatically obtain the DDL statement and make the same schema change in Doris. Quick Start : For MySQL Download the JAR file: https://github.com/apache/doris-flink-connector/releases/tag/1.4.0 Maven: <dependency>
  <groupId>org.apache.doris</groupId>
  <artifactId>flink-doris-connector-1.15</artifactId>
  
  
  <version>1.4.0</version>
</dependency> : For Oracle Download JAR file: , , Flink 1.15 Flink 1.16 Flink 1.17 How to Use It For example, to ingest a whole MySQL into Doris (the MySQL table names start with or ), simply execute the following command (no need to create the tables in Doris in advance): database mysql_db tbl test <FLINK_HOME>/bin/flink run \
    -Dexecution.checkpointing.interval=10s \
    -Dparallelism.default=1 \
    -c org.apache.doris.flink.tools.cdc.CdcTools \
    lib/flink-doris-connector-1.16-1.4.0.jar \
    mysql-sync-database \
    --database test_db \
    --mysql-conf hostname=127.0.0.1 \
    --mysql-conf username=root \
    --mysql-conf password=123456 \
    --mysql-conf database-name=mysql_db \
    --including-tables "tbl|test.*" \
    --sink-conf fenodes=127.0.0.1:8030 \
    --sink-conf username=root \
    --sink-conf password=123456 \
    --sink-conf jdbc-url=jdbc:mysql://127.0.0.1:9030 \
    --sink-conf sink.label-prefix=label1 \
    --table-conf replication_num=1 To ingest an Oracle database: please refer to the . sample code How It Performs When it comes to synchronizing a whole database (containing hundreds or even thousands of tables, active or inactive), most users want it to be done within seconds. So we tested the Connector to see if it came up to scratch: 1000 MySQL tables, each having 100 fields. All tables were active (which meant they were continuously updated and each data writing involved over a hundred rows) Flink job checkpoint: 10s Under pressure test, the system showed high stability, with key metrics as follows: According to feedback from early adopters, the Connector has also delivered high performance and system stability in 10,000-table database synchronization in their production environment. This proves that the combination of Apache Doris and Flink CDC is capable of large-scale data synchronization with high efficiency and reliability. How It Benefits Data Engineers Engineers no longer have to worry about table creation or table schema maintenance, saving them days of tedious and error-prone work. Previously in Flink CDC, you need to create a Flink job for each table and build a log parsing link at the source end, but now with whole-database ingestion, resource consumption in the source database is largely reduced. It is also a unified solution for incremental updates and full updates. Other Features Joining dimension table and fact table The common practice is to put dimension tables in Doris and run join queries via the real-time stream of Flink. Based on the Async I/O of Flink, Flink-Doris-Connector 1.4.0 implements asynchronous Lookup Join, so the Flink real-time stream won't be blocked due to queries. Also, the Connector allows you to combine multiple queries into one big query, and send it to Doris at once for processing. This improves the efficiency and throughput of such join queries. Thrift SDK We introduced Thrift-Service SDK into the Connector so users no longer have to use Thrift plug-ins or configure a Thrift environment in the compilation. This makes the compilation process much simpler. On-Demand Stream Load During data synchronization, when there is no new data ingestion, no Stream Load requests will be issued. This avoids unnecessary consumption of cluster resources. Polling of Backend Nodes For data ingestion, Doris calls a frontend node to obtain a list of the backend nodes and randomly chooses one to launch an ingestion request. That backend node will be the Coordinator. Flink-Doris-Connector 1.4.0 allows users to enable the polling mechanism, which is to have a different backend node to be the Coordinator at each Flink checkpoint to avoid too much pressure on a single backend node for a long time. Support for More Data Types In addition to the common data types, Flink-Doris-Connector 1.4.0 supports DecimalV3/DateV2/DateTimev2/Array/JSON in Doris. Example Usage Read from Apache Doris: You can read data from Doris via DataStream or FlinkSQL (bounded stream). Predicate pushdown is supported. CREATE TABLE flink_doris_source (
    name STRING,
    age INT,
    score DECIMAL(5,2)
    ) 
    WITH (
      'connector' = 'doris',
      'fenodes' = '127.0.0.1:8030',
      'table.identifier' = 'database.table',
      'username' = 'root',
      'password' = 'password',
      'doris.filter.query' = 'age=18'
);

SELECT * FROM flink_doris_source; : Join dimension table and fact table CREATE TABLE fact_table (
  `id` BIGINT,
  `name` STRING,
  `city` STRING,
  `process_time` as proctime()
) WITH (
  'connector' = 'kafka',
  ...
);

create table dim_city(
  `city` STRING,
  `level` INT ,
  `province` STRING,
  `country` STRING
) WITH (
  'connector' = 'doris',
  'fenodes' = '127.0.0.1:8030',
  'jdbc-url' = 'jdbc:mysql://127.0.0.1:9030',
  'lookup.jdbc.async' = 'true',
  'table.identifier' = 'dim.dim_city',
  'username' = 'root',
  'password' = ''
);

SELECT a.id, a.name, a.city, c.province, c.country,c.level 
FROM fact_table a
LEFT JOIN dim_city FOR SYSTEM_TIME AS OF a.process_time AS c
ON a.city = c.city : Write to Apache Doris CREATE TABLE doris_sink (
    name STRING,
    age INT,
    score DECIMAL(5,2)
    ) 
    WITH (
      'connector' = 'doris',
      'fenodes' = '127.0.0.1:8030',
      'table.identifier' = 'database.table',
      'username' = 'root',
      'password' = '',
      'sink.label-prefix' = 'doris_label',
      //json write-in
      'sink.properties.format' = 'json',
      'sink.properties.read_json_by_line' = 'true'
);

Auto-Synchronization of an Entire MySQL Database for Data Analysis

This is a part of the digital transformation of a real estate giant. For the sake of confidentiality, I'm not going to reveal any business data, but you'll get a detailed view of our data warehouse and our optimization strategies. Now let's get started. Architecture Logically, our data architecture can be divided into four parts. : This is supported by Flink CDC, DataX, and the Multi-Catalog feature of Apache Doris. Data integration : We use Apache Dolphinscheduler for script lifecycle management, privileges in multi-tenancy management, and data quality monitoring. Data management : We use Grafana, Prometheus, and Loki to monitor component resources and logs. Alerting : This is where BI tools step in for user interaction, such as data queries and analysis. Data services 1. Tables We create our dimension tables and fact tables centering each operating entity in business, including customers, houses, etc. If there are a series of activities involving the same operating entity, they should be recorded by one field. (This is a lesson learned from our previous chaotic data management system.) 2. Layers Our data warehouse is divided into five conceptual layers. We use Apache Doris and Apache DolphinScheduler to schedule the DAG scripts between these layers. Every day, the layers go through an overall update besides incremental updates in case of changes in historical status fields or incomplete data synchronization of ODS tables. 3. Incremental Update Strategies (1) Set where >= "activity time -1 day or -1 hour" instead of where >= "activity time The reason for doing so is to prevent data drift caused by the time gap of scheduling scripts. Let's say, with the execution interval set to 10 min, suppose that the script is executed at 23:58:00 and a new piece of data arrives at 23:59:00, if we set , that piece of data of the day will be missed. where >= "activity time (2) Fetch the ID of the largest primary key of the table before every script execution, store the ID in the auxiliary table, and set where >= "ID in auxiliary table" This is to avoid data duplication. Data duplication might happen if you use the Unique Key model of Apache Doris and designate a set of primary keys, because if there are any changes in the primary keys in the source table, the changes will be recorded and the relevant data will be loaded. This method can fix that, but it is only applicable when the source tables have auto-increment primary keys. (3) Partition the tables As for time-based auto-increment data such as log tables, there might be less changes in historical data and status, but the data volume is large, so there could be huge computing pressure on overall updates and snapshot creation. Hence, it is better to partition such tables, so for each incremental update, we only need to replace one partition. (You might need to watch out for data drift, too.) 4. Overall Update Strategies (1) Truncate Table Clear out the table and then ingest all data from the source table into it. This is applicable for small tables and scenarios with no user activity in wee hours. (2) ALTER TABLE tbl1 REPLACE WITH TABLE tbl2 This is an atomic operation and it is advisable for large tables. Every time before executing a script, we create a temporary table with the same schema, load all data into it, and replace the original table with it. Application : every minute ETL job : 8 nodes, 2 frontends, 8 backends, hybrid deployment Configuration for first-time deployment : 32C * 60GB * 2TB SSD Node configuration This is our configuration for TBs of legacy data and GBs of incremental data. You can use it as a reference and scale your cluster on this basis. Deployment of Apache Doris is simple. You don't need other components. To integrate offline data and log data, we use DataX, which supports CSV format and readers of many relational databases, and Apache Doris provides a DataX-Doris-Writer. We use Flink CDC to synchronize data from source tables. Then we aggregate the real-time metrics utilizing the Materialized View or the Aggregate Model of Apache Doris. Since we only have to process part of the metrics in a real-time manner and we don't want to generate too many database connections, we use one Flink job to maintain multiple CDC source tables. This is realized by the multi-source merging and full database sync features of Dinky, or you can implement a Flink DataStream multi-source merging task yourself. It is noteworthy that Flink CDC and Apache Doris support Schema Change. EXECUTE CDCSOURCE demo_doris WITH (
  'connector' = 'mysql-cdc',
  'hostname' = '127.0.0.1',
  'port' = '3306',
  'username' = 'root',
  'password' = '123456',
  'checkpoint' = '10000',
  'scan.startup.mode' = 'initial',
  'parallelism' = '1',
  'table-name' = 'ods.ods_*,ods.ods_*',
  'sink.connector' = 'doris',
  'sink.fenodes' = '127.0.0.1:8030',
  'sink.username' = 'root',
  'sink.password' = '123456',
  'sink.doris.batch.size' = '1000',
  'sink.sink.max-retries' = '1',
  'sink.sink.batch.interval' = '60000',
  'sink.sink.db' = 'test',
  'sink.sink.properties.format' ='json',
  'sink.sink.properties.read_json_by_line' ='true',
  'sink.table.identifier' = '${schemaName}.${tableName}',
  'sink.sink.label-prefix' = '${schemaName}_${tableName}_1'
); We use SQL scripts or "Shell + SQL" scripts, and we perform script lifecycle management. At the ODS layer, we write a general DataX job file and pass parameters for each source table ingestion, instead of writing a DataX job for each source table. In this way, we make things much easier to maintain. We manage the ETL scripts of Apache Doris on DolphinScheduler, where we also conduct version control. In case of any errors in the production environment, we can always rollback. After ingesting data with ETL scripts, we create a page in our reporting tool. We assign different privileges to different accounts using SQL, including the privilege of modifying rows, fields, and global dictionary. Apache Doris supports privilege control over accounts, which works the same as that in MySQL. We also use Apache Doris data backup for disaster recovery, Apache Doris audit logs to monitor SQL execution efficiency, Grafana+Loki for cluster metric alerts, and Supervisor to monitor the daemon processes of node components. Optimization 1. Data Ingestion We use DataX to Stream Load offline data. It allows us to adjust the size of each batch. The Stream Load method returns results synchronously, which meets the needs of our architecture. If we execute asynchronous data import using DolphinScheduler, the system might assume that the script has been executed, and that can cause a messup. If you use a different method, we recommend that you execute in the shell script, and check the regex filtering status to see if the ingestion succeeds. show load 2. Data Model We adopt the Unique Key model of Apache Doris for most of our tables. The Unique Key model ensures idempotence of data scripts and effectively avoids upstream data duplication. 3. Reading External Data We use the Multi-Catalog feature of Apache Doris to connect to external data sources. It allows us to create mappings of external data at the Catalog level. 4. Query Optimization We suggest that you put the most frequently used fields of non-character types (such as int and where clauses) in the first 36 bytes, so you can filter these fields within milliseconds in point queries. 5. Data Dictionary For us, it is important to create a data dictionary because it largely reduces personnel communication costs, which can be a headache when you have a big team. We use the in Apache Doris to generate a data dictionary. With it, we can quickly grasp the whole picture of the tables and fields and thus increase development efficiency. information_schema Performance : Within minutes Offline data ingestion time : For tables containing over 100 million rows, Apache Doris responds to ad-hoc queries within one second, and complicated queries in five seconds. Query latency : It only takes up a small number of servers to build this data warehouse. The 70% compression ratio of Apache Doris saves us lots of storage resources. Resource consumption Experience and Conclusion Actually, before we evolved into our current data architecture, we tried Hive, Spark and Hadoop to build an offline data warehouse. It turned out that Hadoop was overkill for a traditional company like us since we didn't have too much data to process. It is important to find the component that suits you most. On the other hand, to smoothen our  big data transition, we need to make our data platform as simple as possible in terms of usage and maintenance. That's why we landed on Apache Doris. It is compatible with MySQL protocol and provides a rich collection of functions so we don't have to develop our own UDFs. Also, it is composed of only two types of processes: frontends and backends, so it is easy to scale and track.

Building a Data Warehouse for Traditional Industry: Hadoop is Overkill

Stories, News & Mentions about DELTASTREAM

Categories

Trending Topics

Stories, News & Mentions about DELTASTREAM

Deltastream’s Stories on HackerNoon

Deltastream's News & Mentions

Categories

Trending Topics