Your dashboards may be cheap. The data movement behind them often is not. Your dashboards may be cheap. The data movement behind them often is not. At first, the dashboard looked harmless. It was a simple operational report. Sales by store.Last 90 days.A few filters. The query ran in seconds. The warehouse computing cost was minimal. From the BI team’s perspective, everything looked efficient. From the BI team’s perspective, everything looked efficient. But when the cloud billing report arrived at the end of the month, something strange appeared. Network transfer charges were increasing. Slowly at first. Then significantly. No pipelines had changed.No dashboards had been redesigned.No new datasets had been introduced. Yet, the system was charging us thousands of dollars for something no one had explicitly planned for. That was the moment we discovered one of the quietest costs in modern analytics: Zombie data movement. Zombie data movement. The Cost Most BI Teams Don’t Track When organizations manage BI costs, they typically focus on two metrics: warehouse compute consumption
query performance warehouse compute consumption query performance These are visible, measurable, and easy to monitor. What many teams rarely examine is data movement between platforms. Modern analytics architectures rarely live in a single system anymore. A typical BI request might travel across several layers: Data Warehouse  →  Semantic Model  →  BI Platform  →  Embedded Application Data Warehouse  →  Semantic Model  →  BI Platform  →  Embedded Application Every time data crosses a platform boundary, it may trigger egress charges. These are costs associated with data leaving a service, region, or cloud platform. Unlike compute costs, egress is rarely obvious during development. Queries succeed. Dashboards load normally. Nothing appears wrong. But behind the scenes, data is constantly moving between systems. And in distributed cloud environments, moving data often costs more than processing it. And in distributed cloud environments, moving data often costs more than processing it. The Multi-Platform Reality of Modern BI Ten years ago, many BI stacks were relatively simple. Database → BI Tool Database → BI Tool Today’s analytics environments look very different. Cloud Storage  →  Data Warehouse  →  Semantic Layer  →  BI Platform  →  Applications / APIs Cloud Storage  →  Data Warehouse  →  Semantic Layer  →  BI Platform  →  Applications / APIs Each layer introduces potential data transfer. Even when the user only sees a simple dashboard interaction, the backend may involve multiple services communicating across infrastructure boundaries. This architectural complexity is one reason cloud analytics spending often grows faster than expected. According to Gartner, by 2027 more than 70% of analytics workloads will operate across multiple cloud services or hybrid platforms, increasing both operational flexibility and infrastructure complexity. According to Gartner, by 2027 more than 70% of analytics workloads will operate across multiple cloud services or hybrid platforms, increasing both operational flexibility and infrastructure complexity. That complexity often includes unnoticed data movement. How Zombie Egress Happens Zombie data movement occurs when data transfer continues without anyone intentionally thinking about the transfer itself. Common triggers include: dashboards querying warehouses across cloud platforms
BI tools retrieving data samples for previews
embedded dashboards serving application users
cross-region semantic models
background refresh queries that move data across services dashboards querying warehouses across cloud platforms BI tools retrieving data samples for previews embedded dashboards serving application users cross-region semantic models background refresh queries that move data across services Individually, each operation may move only a few megabytes. At scale, these movements accumulate quickly. And because the movement is distributed across systems, the cost rarely appears in a single obvious location. A Simple Example Consider a dashboard query that returns 5 MB of results. That sounds small. Now, imagine a moderately active BI environment. 200 users × 40 queries per day × 5 MB returned per query 200 users × 40 queries per day × 5 MB returned per query That equals: 40 GB of data transferred per day. Across a month: ≈ 1.2 TB of data movement Across a month: ≈ 1.2 TB of data movement If that data leaves a cloud warehouse to reach a BI platform in another service or region, it may incur egress charges. If that data leaves a cloud warehouse to reach a BI platform in another service or region, it may incur egress charges. The compute cost for the query might be minimal. But the network cost of moving the results may continue accumulating every day. Why This Often Goes Unnoticed Unlike compute spikes, egress rarely triggers alarms. The system continues functioning normally. Dashboards load quickly. Users remain unaware. The only place the cost appears is in cloud billing reports, often grouped under network transfer or inter-service data movement. This pattern has been observed widely in distributed data systems. As Jeff Dean, Senior Fellow at Google, once explained when discussing large-scale data infrastructure:
“Moving data around the system is frequently more expensive than computing on the data itself.” As Jeff Dean, Senior Fellow at Google, once explained when discussing large-scale data infrastructure: “Moving data around the system is frequently more expensive than computing on the data itself.” In modern BI architectures, that principle shows up clearly. Compute gets optimized. Compute gets optimized. Data movement quietly continues. Data movement quietly continues. The Hidden Feedback Loop One of the most interesting characteristics of zombie egress is that success can make it worse. When dashboards become popular: more users access them
queries execute more frequently
data moves across platforms more often more users access them queries execute more frequently data moves across platforms more often From a business perspective, this looks like a success story. From an infrastructure perspective, it can trigger unexpected network costs. This effect becomes especially visible when dashboards are embedded into customer-facing or operational applications. Why BI Architects Should Care Data movement costs are not just a financial issue. They also signal deeper architectural patterns. If dashboards constantly move large volumes of data across platforms, it may indicate: poorly aligned semantic layers
cross-cloud architectural design issues
inefficient aggregation strategies
dashboards retrieving more data than necessary poorly aligned semantic layers cross-cloud architectural design issues inefficient aggregation strategies dashboards retrieving more data than necessary As Snowflake co-founder Benoit Dageville once noted while discussing modern data platform design:
“The real challenge is not storing the data. It is minimizing how often you have to move it.” As Snowflake co-founder Benoit Dageville once noted while discussing modern data platform design: “The real challenge is not storing the data. It is minimizing how often you have to move it.” Good analytics architecture is often about data locality. Good analytics architecture is often about data locality. The closer the computation happens to the data, the less expensive the system becomes. Reducing Zombie Data Movement Reducing egress often requires architectural thinking rather than query tuning. Some effective strategies include: Data locality: Place BI tools and warehouses within the same cloud region whenever possible.
Aggregation layers: Serve dashboards from summarized datasets instead of transferring raw query outputs.
Semantic model placement: Align semantic layers with the warehouse to reduce cross-platform retrieval.
Caching strategies: Avoid repeated transfers of identical query results. Data locality: Place BI tools and warehouses within the same cloud region whenever possible. Data locality: Place BI tools and warehouses within the same cloud region whenever possible. Aggregation layers: Serve dashboards from summarized datasets instead of transferring raw query outputs. Aggregation layers: Serve dashboards from summarized datasets instead of transferring raw query outputs. Semantic model placement: Align semantic layers with the warehouse to reduce cross-platform retrieval. Semantic model placement: Align semantic layers with the warehouse to reduce cross-platform retrieval. Caching strategies: Avoid repeated transfers of identical query results. Caching strategies: Avoid repeated transfers of identical query results. None of these changes affects the dashboard experience. But they can dramatically reduce the silent costs behind it. The Quietest Line Item in Cloud Analytics When organizations evaluate BI spending, the first focus is usually on compute. Compute is visible. It spikes dramatically. It spikes dramatically. It appears clearly in monitoring dashboards. It appears clearly in monitoring dashboards. Egress is quieter. Egress is quieter. It accumulates slowly across network transfers and service integrations. But as analytics architectures become increasingly distributed, those small transfers can add up to substantial costs. Because in modern BI environments, the dashboard itself might be inexpensive. The data movement behind it rarely is.

This story contains new, firsthand information uncovered by the writer.

The is an opinion piece based on the author’s POV and does not necessarily reflect the views of HackerNoon.

This story contains AI-generated text. The author has used AI either for research, to generate outlines, or write the text itself. 

The Hidden Tax of Cloud BI: Zombie Data Movement Between Platforms

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

AI Is About to Break Your BI Architecture (If You Don't Redesign It First)

10 Great Articles On Data Science And Programming!

10 FinTech Trends in 2021 [Part II]

10 Financial Predictions for 2021 to Protect Your Wealth

13 API Metrics That Every Platform Team Should be Tracking

If You’re Trying to Talk to Everybody, You’re Not Reaching Anybody

AI Is About to Break Your BI Architecture (If You Don't Redesign It First)

10 Great Articles On Data Science And Programming!

10 FinTech Trends in 2021 [Part II]

10 Financial Predictions for 2021 to Protect Your Wealth

13 API Metrics That Every Platform Team Should be Tracking

If You’re Trying to Talk to Everybody, You’re Not Reaching Anybody

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps