'Experience is a Double-edged Sword': Kyle Kirwan, CEO of Bigeye

HackerNoon Reporter: Please tell us briefly about your background.

I’m Kyle Kirwan, co-founder, and CEO of Bigeye. Before starting Bigeye with my co-founder Egor Gryaznov, I was one of the first analysts at Uber and went on to become a product manager in their Data Platform group. I launched the company’s data catalog, Databook, as well as other tooling used by thousands of their internal data users. Those tools helped users with data discovery, quality, freshness, lineage, and other metadata challenges.

Egor and I met on Uber’s Experimentation Platform team which I co-founded. We developed the first set of data pipelines used by product teams across the company to analyze A/B test outcomes.

What's your startup called? And in a sentence or two, what does it do?

Bigeye is a data observability platform that helps data teams make sure all the data their organization depends on is fresh, high quality, and reliable. We automatically collect metadata and use anomaly detection techniques to flag problems in the data and help them solve them before they turn into outages.

What is the origin story?

While at Uber, Egor and I got to work on a bunch of different data platform problems: building analytics products like dashboards and data pipelines, managing the data warehouse and building our own tools for working with data. As the organization blew up in size, we had to keep solving new types of problems to keep everyone fed with the data they needed: warehouse cluster synchronization, pipeline testing, data discovery, understanding which teams contributed what amounts to the infra bill, and the list kept getting longer.

The company was ~200 people when I joined in 2013 and 10,000+ people when I left in 2018. A fairly large portion of the company could run their own queries, build their own dashboards, write their own pipelines, etc. The Data Platform group existed to provide that heavily self-service-oriented model — similar to the “data mesh” concept that’s gaining traction now. My product area within that group was metadata: giving users tools for finding, understanding, trusting, and managing the couple hundred petabytes of data on the platform.

Bigeye is the result of experiencing a bunch of these challenges from different angles — as the analyst, as the pipeline owner, and as the maker of tools — and then talking to people who left the company and missed the tooling they used to take for granted. After I kept hearing, “I really miss tool X,” I went out and interviewed a bunch of data teams and found that freshness, quality, and reliability were at or near the top of everyone’s “pain list” so we decided to start working there.

What do you love about your team, and why are you the ones to solve this problem?

We’re a team of data people creating tools for data people, and everyone is excited about building the tools of the future. Everyone brings a very no-bullshit mentality and a lot of curiosity when it comes to thinking about the things we can build for customers.

Because we’ve been in the trenches, we are a great team for these problems. I’m confident we’re the only ones in the space who have seen both what it’s like to deal with these problems when your data warehouse is a single Postgres box and also what it’s like with thousands of users and tens of thousands of tables replicated across several datacenters.

If you weren’t building your startup, what would you be doing?

I’d probably be hacking together some data tools somewhere else!

At the moment, how do you measure success? What are your core metrics?

Helping customers solve tough problems so they can make magic with data. Customer usage has doubled four quarters in a row and that’s a great indicator of how much our customers are relying on Bigeye. We measure basic usage by how many total observations we make on their data for them. More customers, measuring more data attributes, on more of their data, more often, is what creates more usage.

What’s most exciting about your traction to date?

It’s really all about customers. Every new logo is an immense source of pride. Nothing is better than getting a Slack or a text from a customer saying that Bigeye just caught an issue that would have otherwise taken days to spot and fix.

What technologies are you currently most excited about, and most worried about? And why?

We love using Linear to build Bigeye. It’s really a game-changer for productivity. We also love playing with other products in and around data: I actually just hooked up Confluent, Rockset, and Redash last night to get a toy real-time analytics dashboard up and running. We also already run a bunch of other tools internally like dbt, Astronomer, and some other stuff. The day I have to completely give up playing around with new data products will be a sad one.

I don’t know that I’m worried much about any new technologies. One thing I do worry about is jargon, and how it can get deliberately complicated by vendors to confuse customers and better position their own products. I think it’s a net negative for the industry because it slows down our collective understanding in exchange for gains that go to a single company.

What drew you to get published on HackerNoon? What do you like most about our platform?

I love a publication that caters so much to a technologist audience. As tech and software engineering and data science gain more awareness in general society I think it’s important to have a publication that maintains such a focus on practitioners.

What advice would you give to the 21-year-old version of yourself?

You actually do love backpacking, you just need to go to Yosemite to realize it. Go! Go now!

What is something surprising you've learned this year that your contemporaries would benefit from knowing?

Experience is a double-edged sword. You get the benefit of having ingrained patterns that help you short-circuit product challenges, or key in on things customers say that implicate solutions they aren’t aware of yet, and things like that. But it also comes with your own biases based on the environment you were in when you assimilated those patterns into your thinking. That’s a tradeoff you’ve already made and can’t entirely undo.

The earlier you can understand your own bias /precision reality, the better you can help your customers.