Setting goals for product features is an art in product building. A product is successful only if it achieves intended objectives or goals. Defining and measuring the product’s or its features’ success need a good understanding of company goals, business, product vision, strategy, roadmap, its users, end customers, and partners. Hence, it is also a standard question in PM interviews at all levels, including the Product Execution Interviews (at Facebook).
Here is a step-by-step guide about setting product goals and their success metrics before you build any product. It is also a treasure for PM interviews.
Goal Metrics Interview Problems :- How would you go about setting goals for Instagram hashtags? How would you measure success for IG stories? How would you set goals for Instagram Reels? What goals and success metrics would you set for buy & sell groups? How would you set a goal for Facebook Lite? What top 3 metrics would you track for Quora? What should Uber eats goals be for the next 12 months? What would be the goals & success metrics for video conferencing app (Hangouts, Zoom)?
So what does product success look like?
A product goal describes a long-term objective or future state of the product. Product metrics are needed to track the success of the product against its goals. These metrics help to identify existing gaps, plan a roadmap and guide the prioritization of the next features and enhancements. Metrics are extensively used during
Leadership approvals (business case)
Product launches (as success criteria)
Annual, quarterly plannings (as KPIs - Key Performance Indicators or OKRs - Objective Key Results or OSTMs - Objectives Strategy Tactics & Metrics)
Sprint exercises (as acceptance criteria)
To measure the success accurately, goals need to be SMART - Specific, Measurable, Attainable, Relevant, and Time-bound.
What are the different classifications of metrics ?
To measure the success of a product, three types of metrics are used -
Metrics are also classified as -
Moreover, leading metrics (leading indicators) are those which are forward-looking and can represent future outcomes. In contrast, lagging metrics (lagging indicators) represent current and past state and are not good predictors of the near future. E.g., When measuring sales, Revenue per user is a leading indicator while churn rate is a lagging indicator. Dropping revenue per user is an urgent sign of sales reduction, while a high churn rate of customers over a longer period will eventually impact sales.
Now, let’s discuss how we set goals and metrics to define a product’s success.
It is important that you understand the product, company, and problem space. Ask relevant questions to clarify the situation and develop a common understanding.
CLARIFYING QUESTIONS TO ASK :- What is the product and the company ? What is the stage of the product in its lifecycle? Should we focus on any geography or think globally ? What is the competitive landscape ? Is there a particular company strategy to build new goals ? What are the key user values and actions ? Do we have any particular customer segment or situation like Covid I should focus on? Is there any specific time frame for goals ?
Note:- Not all questions need to be explicitly asked. You may make some relevant assumptions and state the same. E.g., if the product is Facebook Live, you can mention - “My understanding is that Facebook Live is a matured product and is aimed at driving engagement for global audience.”
Reiterate the mission and vision of the company and/or product. This is helpful in building the right goal strategy; say - “Before I build an approach, let me touch upon the mission for Facebook ”
MISSION :- Facebook - Make world more connected and help people build communities Instagram - Allowing people to capture and share moments of their life To bring you closer to the people and things you love Google - Organize the world's information & make it universally accessible & useful FB Watch - allows users to view videos from friends, groups & thus bring them together
Product goals are tightly coupled with the users in almost all modern businesses. Identifying different users and their journey help to build a comprehensive list of user metrics and possible goals.
Users can be Supply-side - Demand-side; Primary users - Secondary users; Internal users - External users; Buying customers or Users using the product.
E.g., Instagram’s users are
User Journey :- Discover > Interest > Desire > Reach > Find > Access > Learn > Test > Use > Complete > Validate > Share
E.g. User journey of a traveler:- Deciding Destination > Plan Itinerary > Activities > Express and Share memories
USERS :- Artists, Content creators, Advertisers, Young Adults, Parents, Kids, Elders, Students, College goers, Religious, Working professional, Traveler, Affluent, Price Conscious, Risk Averse, Influencer, Follower, Volunteers, Activists, Conservatives, Digital Nomad, Hiker-Traveler, Educators
Detailed user segmentation is generally not necessary, but it is still best to ask if goals should be for any specific segment - read segmentation.
Now that you have better clarity about the mission and the users, let’s build a comprehensive list of relevant metrics.
Generally, metrics are measured across the entire user funnel -
At least two or three metrics should be identified across the funnel so that we can accurately prioritize the most appropriate metrics in the next step. It is also important to use an appropriate time frame for these metrics, which generally depends on the company’s performance review cadence (weekly, monthly or quarterly).
The metrics should be measured across all relevant users, supply, demand, business, and platform (whatever applicable) - consider 2X2 of these against the funnel metrics.
E.g., For Airbnb, the key metrics are
Note:- Many of the metrics are heavily inter-related e.g., Facebook makes revenue via ads (monetization), and ad revenue is directly proportional to the engagement and conversion metrics of users like clicks, subscriptions, and also their retention on the platform
In the previous step, we defined various metrics for each funnel stage that indicate movement towards multiple possible goals. Now, it’s time to build a goal strategy by prioritizing a goal and selecting the most relevant and impactful metrics.
First and foremost, the product lifecycle stage (new, growing, mature) is used to identify which part of the funnel to focus on as a key goal. Select the most appropriate goal from the funnel (awareness, acquisition, adoption, engagement, conversion, retention, monetization, satisfaction) which complements the company and product strategy.
For e.g. -
North Star Metric is the most important broad indicator (leading metric) to track the goal. It indicates whether the product is benefiting the users as well as impacting the business (i.e., satisfying core product value). Based on the strategy, different companies, even in the same industry and lifecycle, optimize for different goals and metrics. For the interview, it is recommended to specify one North star metric, and two to five supporting metrics that relate best to your identified goals. It is also advisable to provide a couple of counter metrics or guard rail metrics to track any counter affects and risks associated with the product or feature.
E.g., For the Instagram Reels feature, the company’s goal is engagement, and it is measured by the primary metric (North Star Metric or NSM) of Daily Active Customers actively engaging with Reels.
North Star Metrics - Daily Active Customers, engaged with Reels (with >15 secs view time)
Product health metrics - No. of User Engagements on Reels (likes, shares, comments), No. of Reels Created, No. of Customer Views, $ Revenue from ads
Counter Metrics - User Session time or Time spent on Reels vs. other areas of Instagram
NORTH STAR METRICS :- Instagram - Weekly Active Customers (engagement) Facebook - Monthly Active Customers (engagement) Uber - Bookings completed per week (conversion) Airbnb - Nights booked per week (conversion) Quora - Number of questions users answered (engagement) Whatsapp - Number of messages sent weekly (engagement) Youtube - Total engaged watch time (watch time >10 secs) (engagement) Netflix - Number of subscribers with watch time >1 hours in the month (engagement) Salesforce - Number of records created per account (engagement) Adobe - Revenue from subscribing customers (monetisation) Walmart - Purchases per week (conversion) Amazon - Purchases per Prime subscribers (custom strategy) Prodbee - Number of page views (acquisition)
Pro Tips :-
- Keep the prioritized metrics as simple as you can
- As companies evolve from new to growing and matured, their goal shifts in the funnel from Acquisition to Monetisation and Satisfaction
- New products have simple metrics around customer acquisition; mature products have more detailed/custom metrics like number of engaged users >30 sec watch time
- Do not use the vanity and non-actionable metrics
- Try to use absolute metrics than rates to avoid the base effect
- Do not forget the timeframe of measuring the metric (e.g., no. of new sign-ups weekly)
- Avoid competition benchmarking metrics like market share unless necessary. For e.g., new business launch
- You can also measure metrics as target vs. actuals like quarter to date
It is recommended to critique your metrics by discussing any broader risks and foreseen fallback. You can discuss the counter metrics to highlight cannibalization, overuse, and customer satisfaction issues. E.g., If a product feature is newly launched, it is important to closely track the customer reactions on social media, customer support issues, bugs, resolution time, and various platform performance indicators like downtime, # bugs, average load time, etc. Tracking edge-cases or failure rates such as no matches, no results page, and inactive users are also key metrics to measure the success of products depending on their maturity and business relevance.
Note:- In interviews, goal setting question is often followed by another Product Execution Question like -What would you do as PM if this product metric X goes down by Y %?
(see Solving a Product issue )
"Happy Product" means "Happy users' & "Happy numbers" Still have questions or need any help on Goals and Metrics ? Drop a comment If you liked the article, please support by sharing this story or https://prodbee.com/
Also Published Here