How DAGs Grow: When People Trust A Data Source, They'll Ask More Of Itby@kyleweaton
220 reads

How DAGs Grow: When People Trust A Data Source, They'll Ask More Of It

by Kyle Eaton6mDecember 9th, 2020
Read on Terminal Reader
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

When people trust a source of information, they tend to ask more questions of it. Data teams spend their time asking questions and building infrastructure to make analysis repeatable. When the same questions have repeated value, their supporting DAGs tend to grow in three specific ways. When you’re building a data system, you start noticing that you spend a lot of time on maintenance: bug fixes and root cause analysis. This turns into a steady, compounding creep in the maintenance time that you're putting into the project. We call this phase the “miracle of software”

Company Mentioned

Mention Thumbnail
featured image - How DAGs Grow: When People Trust A Data Source, They'll Ask More Of It
Kyle Eaton HackerNoon profile picture
Kyle Eaton

Kyle Eaton

@kyleweaton

Community Manager of Great Expectations

About @kyleweaton
LEARN MORE ABOUT @KYLEWEATON'S
EXPERTISE AND PLACE ON THE INTERNET.
L O A D I N G
. . . comments & more!

About Author

Kyle Eaton HackerNoon profile picture
Kyle Eaton@kyleweaton
Community Manager of Great Expectations

TOPICS

THIS ARTICLE WAS FEATURED IN...

Permanent on Arweave
Read on Terminal Reader
Read this story in a terminal
 Terminal
Read this story w/o Javascript
Read this story w/o Javascript
 Lite