This article provides definitions on what key metrics are, a framework for categorizing them, and recommendations of four basic rules to set useful metrics, using real-life examples.
Show me the data
Asa Product Manager, you might have been in this situation: you are in a middle of a meeting with C-level, and they want to know how the new feature you have been working on helped the company goals. You start explaining how it’s improving the user experience, making it much easier to use and that it will attract new customers, but Management is not impressed: they want numbers.
As the old controversial motto says:
“If you can’t measure it, you can’t improve it.” — Peter Drucker
That’s when the importance of data and metrics come in to place.
What you will find in this article:
- Definition of a metric
- A real-life case of focusing on the wrong metric
- Examples of Engagement and Growth metrics for companies such as Twitter, Facebook, and Youtube
- Four basic rules to follow for having useful metrics
What is a metric
But let’s start by defining what a metric is.
If we look at Investopedia financial definition, we have that metrics are “measures of quantitative assessment commonly used for assessing, comparing, and tracking performance or production. Generally, a group of metrics will typically be used to build a dashboard that Management or analysts regularly review to maintain performance assessments, opinions, and business strategies”
In other words, metrics are numbers saying what is going on with your product.
You might also have heard of Key Performance Indicators (KPIs) or Key Value Indicators (KVI) if you are an adept of the Evidence-Based Management from scrum.org. If you are working for a tech company, some examples of metrics such as Monthly active users, Returning users, Churn, App store reviews might be familiar to you.
Example of metrics:
- Monthly active users (MAU) and all their variant such as Daily Active User (DAU)
- Returning users
- Impressions
- Earned Media Value
- Revenue
Anything quantifiable and useful to understand if your business is going in the right direction could be used as a metric. Metrics can vary a lot according to your industry, your business, and even the stage of your product.
Let’s say you just launched your new Android app: it’s an MVP, and you want to gather as much feedback as possible to reiterate and add value for the end-users. You focus on the number of downloads: you want your app to be seen by as many people as possible. After some time, you think: what’s the point of people downloading my app if they don’t end up using it. From concentrating on the “app download metric,” you switch to usage metrics such as “number of logins,” “time spent by day,” etc. Once you manage to get your users to play with your app and interact with it, you can focus on maximizing value. From “usage” metrics, you switch to “satisfaction/happiness” metrics such as Net Promoter Score (NPS). Having in mind that the switch from one metric to another doesn’t mean you should forget about the previous ones; you could focus on happiness but still track and care about downloads and usage.
If you focus on the wrong metrics — s#$ can happen & fast
I lived the experience of being in a startup where our primary metric (our KPI) was revenue. We would do any promotion to generate sales, promo-code would rain, we would sell at a loss reducing the price drastically to be competitive. What only mattered was this: increasing the revenue and showing that our market was big.
The only thing that matters was increasing our revenue (at any cost)
Photo by Erik Mclean on Unsplash
Until we got new investors that thought our business was old enough (meaning 2+ years in the startup world) to start thinking about making money. Our new goal: being profitable!
As profit became our main KPI, we had to drastically change our strategy since our primary revenue driver was making us lose money. We had to cut sales to improve our margins and our capacity to make money.
When our new KPI switched to profitability the company had to change their strategy drastically
So the hardest part about metrics is defining which one would be the best to point out if you are on the right track or not.
Your role as a Product Manager: defining success through metrics
If you are an adept of Scrum and familiar with the Product Owner role (sometimes described as an Agile Product Manager) you know that the primary responsibility is to “maximiz[e] the value of the product resulting from the work of the Development Team” (scrum.org)
As a Product Owner, if you want to be successful, you must define what you want to hit. You create a vision for your Product, you set up goals to achieve, and you define what success would look like in terms of metrics.
Photo by Turnaroundscience
Let’s say you work at Twitter, and one of the forms you define success is by stickiness — if your users keep coming back and how relevant is your app for them. Data from your analytics team show that users who follow 10+ users are 40% more like to still be active in a month compare to users who don’t (they follow 0 to 9 users). As you want to increase stickiness, you might think of a new feature that encourages to follow users: maybe you could rethink the onboarding when a user creates an account and force them to follow a minimum of users to continue. Your new goal and metric could be “Decreasing by 20% the number of new users who follow less than ten users”.
Growth metrics and Engagement metrics are the most used in Tech companies
Metrics can be categorized in all sorts of ways: growth, engagement, retention, etc. If you look at most famous current Tech companies like Twitter, Facebook or Youtube they would mainly focus on 2 types: Growth metrics and Engagement metrics.
Categories of metrics:
- Growth and activation
- Engagement
- Retention
- User happiness
- Revenue
Growth metrics are used to examine a company’s historical growth and hopefully provide clues for the future. (ycharts.com)
Engagement metrics are used to track specific behavior from the users with your product, measuring how the user interacts with your product.
If we look at concrete examples for the previously mentioned companies, we would have
Twitter Growth metrics
- Total new users per month, week, day
- Monthly active users, daily active users
- Activated users
Twitter Engagement metrics
- Multiple logins per day
- Time spent
- # Tweets by users
- # like, retweet, follow by users
- # message sent
Youtube
Growth metrics
- Monthly/daily active users
- Total new users
- Activated users
Engagement metrics
- Video views (at least 30s)
- Average viewing time
Photo by Datapine
What makes a good metric:
Now that we have defined the most used metrics, let’s see what the steps to chose useful metrics are.
Those are simple rules to follow when setting up metrics:
- Understandable
- Rate or Ratio
- Correlation
- Changeable
Understandable: You should easily be able to explain the metrics; it has to be something simple.
Rate or ratio: The metric should be a portion of a more significant number. For example, at Airbnb, the total number of nights booked is a number too big. If we look into the average number of nights booked per person, then we get to a metric that is more insightful for the team.
Correlation: You must take into consideration the correlation when looking at metrics. Sometimes there is a pattern that you want to follow, but in reality, assuming things that are not true. A simple example is the number of deaths in the swimming pool and the sales of electric fans. They follow the same pattern, but they are not correlated to each other: the common denominator is that they both happen in summer.
Changeable: You need to be able to have an impact on your metric. The metric has to be able to change; if no, there is no point in tracking it. Let’s say you are an e-commerce platform where your users only log in once a month to check the offers. It might be unlikely that you managed for them to connect more often. What you should focus on instead is trying to make them spend more every time they come to your e-commerce.
Congratulations if you made it this far! This article is just a beginner intro into the world of metrics for Product Manager. I plan to write more in-depth articles about popular frameworks such as HEART (Happiness, Engagement, Adoption, Retention, Task Success) or AARRR (Acquisition, Activation, Retention, Referal, Revenue) and Evidence-Based Management.