The Newest Frontiers on the Modern Data Stack

In the past few years, there’s been an explosion of tools in the data ecosystem, allowing companies to sync, store, transform, serve, and analyze data cheaper and faster than ever before. The rich ecosystem of tooling surrounding this movement has been referred to as the “Modern Data Stack”. On the left side of the diagram, you have your data sources — this can be your application’s production database, as well as SaaS tools like Shopify, Stripe, and Google Sheets where customer data can live. Next, you’ll use tools such as Fivetran and Stitch to load data from these sources in your data warehouse. You may also use tools like dbt to transform your data to make sure it’s in the right schema for further use. Next, you have your data warehouses — hyper-efficient and scalable centralized data stores for all your data. The robustness and speed of modern-day data warehouses have played a pivotal role in driving the innovation of the Modern Data Stack. Finally, on the right-most side of the stack are tools that help you extract insights and take action on your data. Tools like Tableau and Sisense have been around for a long time to help you analyze and visualize data. Tools like Hightouch and Census are relatively newer, helping you sync data from your warehouse back to SaaS applications like Mailchimp and Salesforce. Finally, we have the newest frontier on the Modern Data Stack which is data-driven operations. Tools such as live in this category to help you not only analyze your data but take action on it. Let’s take a look at some concrete use cases. LogicLoop Data-driven operations use cases alert when a condition of your business operations needs attention: Business Operations Alerting — — create a case when a suspicious transaction occurs on your platform Fraud monitoring — email an Account Executive if a user is using advanced features of your product Sales opportunities — ping a product manager if new user sign-up counts are low Product usage monitoring - send a manager a report if customer support response times are too slow Team productivity stats send customers outreach communications when certain conditions are met: Customer Outreach — — send a welcome email when a new user signs up for your platform New user onboarding — send a congratulatory text when a user hits a certain milestone on your platform User milestone notifications — send a reminder email if a user has not been active on your platform for more than a month User churn notifications — send a weekly summary of their usage stats on your platform Weekly usage reports automate the next step in a process when a business event occurs: Automation — — automatically order more inventory when an SKU count is low Inventory management — automatically increase or decrease ad spend based on performance Manage advertising spend — automatically create a Stripe subscription for new users who sign up Billing alert when a data condition is not met or an external API is down: Systems Alerting — — alert when a data field is NULL or contains an inappropriate value Data conditions — generate a PagerDuty alert if an external API you rely on is down External API Once you see all the workflows you can build with data-driven operations, you won’t be able to un-see them. Why is this interesting now? These types of workflows are very common and surely have been built by many companies over the years, so why is this significant? Primarily because in the past, many of these workflows needed to be built and maintained by engineers, whose time and resources are very expensive. With the rise of the modern data ecosystem, non-technical professionals like business operations analysts can now unlock these workflows on their own with some SQL knowledge. This allows their company to grow and scale their operations faster and increase revenue-generating touch points without needing more engineering headcount. Data-driven operations tools like LogicLoop can help you quickly set up these workflows to help your company scale. Conclusion Data is becoming more and more accessible and reliable, and non-technical employees are increasingly becoming more data literate, creating an opening for a business operations platform to be built directly on top of the data warehouse. While the data-driven operations space is relatively new, it’s growing quickly along with the rest of the ecosystem. With all the innovation that has occurred in the data space, it’s a great time to continue to push the frontier of what’s possible. Here’s to seeing this category expand widely into the future! Also published . here