How To Use OKRs For Your High-Growth Startup

For the context, Airbyte is an open-source data integration platform. Our goal is to commoditize data integration. In January, we shared how we were thinking about OKRs, along with our OKRs for Q1 2021. So we wanted to give some updates about them, and how they have evolved for the 2nd quarter. 

Our focus for 2021 is to become the open-source standard for replicating data. This entails three overarching goals:

• Making Airbyte just work whatever your data infrastructure, volume and connector needs.
• Building the largest developer community for data integration. We envision that most connectors will be built and maintained by the community eventually, because we will have made that so simple with our low-code framework.
• Making Airbyte so easy to use in a production context that Airbyte becomes the new standard for data teams to replicate data.

Let’s see how this translates itself into our first two quarterly OKRs.

How We Performed on Airbyte’s OKRs for Q1 2021

1. O: Growing Community Love

What is community love? We’re still big fans of Orbit’s definition for it. Love is a member’s level of engagement and investment in the community. Someone with high love is highly active and plays key roles in the community, like contributing, moderating, and organizing.

Let’s first look at GitHub Stars

In this chart, we’re comparing Airbyte with other famous open-source projects around data integration: DBT and RudderStack. Our growth rate (Airbyte in red) is a huge validation that we’re not the only ones to believe that data integration will be solved with an open-source and community approach.

GitHub stars are good awareness metrics, but they don’t mean that you actually have community adoption or contribution. We need to look at other metrics for that:

Overall, we outperformed our Q1 OKRs for community love, even though we set aggressive goals. This is still the very beginning of our journey, but this was extremely encouraging for all the team. We strongly believe we can commoditize data integration through our growing community.

2. O: Growing Production Usage

We call “activated users” users who have deployed Airbyte, connected a source, a destination and synced data successfully from this source to this destination.

We call “prod users“ users who have been syncing data more than 5 times in the past week and 5 times in the week before.

Here’s a chart showing the evolution of activated users and prod users during Q1.

We don’t publish the number of prod users we have yet, but you can see that the conversion from activated to prod users is growing with time, which is what we want to see.

But, is the usage of Airbyte growing among prod users?

If we had to follow only one graph, it would be this one. It accounts for both prod user growth and usage growth within prod users.

Here’s the usage growth in terms of sync per prod user:

Overall, this was exactly what we wanted to see. Teams start by testing Airbyte for a few days or weeks, before expanding their usage to other connectors.

3. O: Becoming a Reliable Standard

Airbyte can only become the new standard if connectors are reliable. You could consider that a “sanity” metric — in the sense it is not related to some growth metrics -, but it is actually where almost all of the engineering work goes. The more users use Airbyte, the more edge cases connectors get exposed to. It is a thousand-paper-cut problem, where every user comes with their needs in terms of usage, data and volume. The more users we have, the less reliable connectors can appear, and we have to seize these opportunities to strengthen them.

The metrics we’re looking at in this case are the percent of failures at sync attempts:

We launched on HackerNews on January 26th. That’s when we gained a lot more users at once and got exposed to a lot more use cases. During the whole month of February, we worked on strengthening our connectors, and you can see in this chart how it paid off. Our KR was 5% of failures by the end of the quarter, and this is something that we will keep working on.

Some other metrics we wanted to track:

• KR: Response time to any message on Slack or GitHub — our goal was to reach 30 min by end of Q1 2021.
• KR: Time to high bug resolution — our goal is to reach 1.5 days by the end of Q1 2021.

In the end, we couldn’t really measure those 2 metrics. But the overall response time to any message on Slack was about 1–2 hours.

4. O: Building the Dream Team

We strongly believe in talent density, and that it’s better to have one stellar colleague than 5 average ones.

• KR: 2 A+ engineers => 3 engineers will be joining us in the next few weeks.
• KR: 1 senior developer advocate => Abhi will be joining us soon!

Our Q1 Milestones

Now that we have seen how we performed on our OKRs, how did we perform on the milestones?

Community efforts

January:

• Hard launch on HackerNews
• Building tutorials to improve the developer experience (DX) in building their own connectors, or editing pre-built ones => this is still a work in progress.

Product engineering efforts

One thing we didn’t anticipate is the toll providing great support would take on our engineering velocity. Even though we had great output, we were not able to deliver on all the milestones we had intended.

For our core platform:

• Integration in data stack with DBT and Airflow => delivered, although we still have a lot on DBT’s front!
• Core upgrade strategy => delivered!

For our connectors:

• Strengthen our connectors so all our connectors are A+ => we started certifying our connectors against a set of best practice, and you can now see the health status of our connectors.
• Schemas migration management => reprioritized
• Seamless OAuth support => reprioritized
• More high-level abstractions to build connectors more easily => ongoing effort!
• An MVP for CDC (Capture Data Change) => delivered!
• Connector upgrade strategy => delivered!
• A public dashboard showing the stability (failure rate) of all our connectors => delivered!

Our New Q2 OKRs

So what about the next quarter? Doing OKRs is actually a great learning opportunity enabling us to make better estimates every time. This time, we have experience on how much time providing a great support experience takes in engineering time. So we can plan accordingly.

For Q2, we kept the same objectives but changed some KRs that we’ve put in bold.

O: Growing Community Love

• KR: Active Slack users (Q1/21: 350, Q2/21: 600)
• KR: GitHub stars (Q1/21: 2k, Q2/21: 4k)
• KR: Issue contributors from start (Q1/21: 125, Q2/21: 250)
• KR: PR contributors from start (Q1/21: 25, Q2/21: 50)
• KR: Connector Contributors (Q1/21: 10, Q2/21: 30)

O: Growing Prod Usage

• KR: Prod users
• KR: Active connections per prod user
• KR: # connectors (Q1/21: 56, Q2/21: 90)

O: Becoming a Reliable Standard

• KR: % failure at attempts
• KR: average throughput of connectors
• KR: support replicating large databases in X minutes

O: Building the Dream Team

• KR: 2 A+ engineers
• KR: 1 dev evangelist (to be confirmed) + 1 operations manager

Our Next Q2 Milestones

How does this translate into milestones?

• Make Airbyte the easiest way to create line-of-business connectors with our low-code solution for creating connectors quickly and more reliably.
• Support custom DBT models.CDC for all major database sources.
• Mature handling of (large) production data sets.
• Production-grade single node support (across platforms): creating solid AMIs, systemctl, etc., with less setup.
• First-class support on K8s.
• OAuth support for connector authentication.
• “Automatic” Schema change handling.
• Support for data lake use cases.

So…a lot of engineering milestones! And they can be accomplished as we grow our engineering team.

Let’s see how we perform in 3 months!

Previously published at https://airbyte.io/articles/our-story/how-we-performed-on-our-q1-okrs-and-the-goals-for-q2/