Product Management — How we do what we do

Where I try to make sense of the dark art of being at the intersection of many different disciplines to create world-class experiences for Internet Economy products.

I’ve battled with explaining what I do every day for a while. The Mini-CEO of the Product, the Good PM/Bad PM and so many other great examples have framed my thinking, and helped me come up with my own definition. This helps me understand the type of problems I solve with my teams every day and, as such, our lacks and faults, and how fundamental they are to what we’re creating.

I’ve divided the practice of Product Management into 4 operational planes.

From the ground up:

a. Context > b. Hypotheses > c. Outcomes > d. Process

I then categorise each of those planes with 4 fundamental attributes:

1) The Big Question – What is the fundamental thing we’re trying to answer in this plane?

2) Inputs – What do we need to know to answer the big question?

3) Outputs – What do we get out of answering it?

4) Activities – What do we have to do in order to answer it?

So starting from the foundations, let’s move up the product manager role.

Context

The plane of context is about fully understanding why you’re doing what you’re doing.

Whatever problem you’re solving, you need to:

1) The Big Q – Start with why

Why are we solving this problem?

Before buying a new house in an unknown neighbourhood, you need to know as much as possible about its intrinsic and extrinsic characteristics: the location, transportation options, nearby schools, criminality, amenities, does it have a garage, do people complain about the noise, is there enough parking, etc.

Similarly, a great product manager collects and explores the collective insights of her organisation to understand the context of where they’re building their product to nail why the problem needs to be solved.

Because maybe… it doesn’t.

2) Inputs – what do I need to know

How do you get to know the location where your product will be built?

I’d say you need at least a good grasp of: your Organization, its Vision and Strategy, the Market, your Users/Customers, the Technology and the Business Model. Talking to people, getting out of the building, feeding off of customer research, being aware of what competitors are building and the potential business models you could explore, but also the technical possibilities and constraints you’re faced with, will allow you to make the decisions you need to make.

3) Outputs – what have I learned

One of the most important jobs of a product manager is to turn the why into the what. He does that by compounding the insights from these dimensions, usually the domains of a business function (Growth, Commercial, Engineering, Customer Support, UX) into cross-cutting insights, quantitative and qualitative analysis and sometimes ambitious ideas that can transform the product and were lost in the noise or knowledge silos of the organisation.

The other output is your positioning. Only by fully understanding the context and why you’re solving the problem, can you clearly and confidently state what you’re building, and, sometimes more importantly, what you’re not. Ryan Singer explains it better than I could ever hope to.

4) Activities – What do I need to do

Research – tons of it. Company Strategy. Competitors. Market. Business Models. Lean, Agile, Kanban. User Research, Kano Studies, Jobs To Be Done. GMV, CPC, CPM, LTV, CAC, SaaS. Microservices, React, Docker, Machine Learning, Experimentation. Kubernetes and AWS. Content. Localisation and Internationalisation. SEM and SEO. You don’t need to be an expert in any of them, but you need to know enough to make informed decisions and drive insights. Talk to people, get out of the building, find out why you’re solving this problem.

Hypotheses

1) TBQ — What Problem are we solving? Is this a real problem?

Problem definitions are usually broad, so when moving from the context plane into the hypotheses domain, we start to define what we believe in. Just know why we’re solving something doesn’t give us enough boundaries to define the problem. So we hypothesise:

“Based on insight X, I believe that change Y will produce impact Z”.

It’s important to define the problem in these terms, because it forces us to confront the fact that we’re attempting trying to model the world with our knowledge. It’s a belief, an opinion about how the world works, based on an insight, not a statement of fact or a pure guess. You’ll be surprised at how often you’re wrong.

2) Inputs — Insights, Ideas, Customer Needs

The best way to collect insights is through a thought process at the right level of modelling the world. Finding patterns and similarities, analogies, weird customer behaviours, unexpected data patterns after a serendipitous piece of feedback or an interview where someone struggles with a facet of the product.

Clay Christensen said that “Questions are places in your mind where answers fit” — be curious enough to find the unexpected insights and brave enough to put them to the test.

3) Outputs — Solution attributes, Acceptance Criteria

“We will test this by assuming the change has no effect (the null hypothesis) and running an experiment for 1 week. If we can measure a Y% statistically significant change in [metric] then we reject the null hypothesis and conclude there is an effect.”

This is the standard hypothesis evaluation mechanism. The goal is not to get it right every time, but to scientifically understand what we want the outcome to be. How we think of success might change over time. but it’s essential for us to be able to extract focus out of our experiments. As we move up the planes, the picture should be getting clearer. By failing to see the effect of a given hypothesis in an experiment, one should use that to refine what the solution should be. Similarly, by seeing an experiment succeed, one should not rush to think he has conquered the world, but merely been able to establish a causal relationship between an insight and an outcome.

4) Activities

Experimentation is the tool of choice for the hypothesis plane. It’s not (just) about building MVPs, though. JTBD interviews, Design Sprints, rapid prototyping, ODI, applying the Kano model and other types of user research can prove or disprove a hypothesis.

If you can’t quantify your outcome and are still in qualitative land, define what you expect it to be upfront. This is a tool to fight confirmation bias and avoid only seeing the results you want to see.

What if disproving your idea could take only 5 seconds?

Find the activity that‘s right for your problem. The goal is to really understand it.

(For a handy list of human biases, do check this out.)

Outcomes

1) TBQ — How will we know we’ve solved it?

The next plane, when the hypothesis has been properly defined, is the plane of outcomes. How do we know we’ve solved a problem we believe is real? Will we consider our job done when we develop a cheap MVP that does the trick? Or will we iterate on it until we’ve found the local optimum? If there’s another maximum elsewhere, will we attempt to find it? How can we avoid premature optimization and building a death star?

The question is how we define success for ourselves. What’s our exit criteria. It’s turning the attributes of our solution into goals we can track.

2) Inputs — Solution attributes, acceptance criteria

Knowing the hypothesis clearly, you want to formulate preliminary acceptance criteria that will validate it. What’s the Riskiest Assumption to Test? What are the primary attributes of that assumption? How do we design them to be understood? What’s the impact we need to see on user behaviour to accept them – Our Overall Evaluation Criteria?

It’s natural to start with a set of attributes, but not have value assigned to them – no exit criteria. This is where you need to manage for outcomes.

3) Outputs — Metrics and KPIs, Lessons learned

Define clear attributes for success.

Let’s say you’re introducing a friend discovery service to optimize on-boarding in your social network. What’s the impact you expect to make? Are you measuring leading (number of friends added in X days) or lagging (retention, engagement) indicators – have you established a causal relationship between them? Do you know what effect you need, to obtain statistical significance? If you go from 5 to 6 friends in 7 days, does that make a difference in your star metric? How about 5 to 8 friends? 5 to 10?

Managing for outcomes means understanding what the solution needs to deliver and what are the key attributes it needs to possess to deliver them.

4) Activities

Using the hypothesis kit above gives you the minimum expected impact you can measure. It still doesn’t solve the issue of statistical significance vs product significance. One can work for a year to deliver a 0.02% increase in conversion rate. Was it worth it?

Define realistic stretch goals — watch Rick Klau’s OKR presentation for some guidance. If you need a 5% increase to deliver impact on your star metric over the course of year, but it’s likely that your product increment won’t be able to deliver anything over 2%, set your goal at 2.5%. Force your solution to creatively validate your ability to achieve the end goal, without demoralising your team. The OKR principle of 70% achievement is always a good start. Managing for outcomes means really understanding the metrics and how the solution will deliver them. Don’t sell your efforts short. Drive the impact.

Process — or how we Get Things Done

1) TBQ — How are we solving it? When are we solving it?

A lot of Product Managers decide process first, and then let that drive the outcomes. I argue process is an important plane to manage in, but it should be driven by the outcomes you want to achieve.

Additionally, these top 2 layers are where a lot of Product Managers get stuck. They focus on Outcomes and Process without really understand what or why. The backlog is perfectly groomed, Story Point delivery is 20/20, so they produce beautifully crafted roadmaps. But when we look back on the impact of what we’ve delivered, we haven’t moved the needle at all.

Did you really need a Product Manager for that?

That’s not to say these aren’t important. Giving your team a sense of progress, predictability and purpose are what you want to draw out of process. Setting clear deadlines allows us to manage scope, provides autonomy, stimulates mastery and creativity and manages expectations.

2) Inputs — Metrics and KPIs, Constraints, Lessons learned

How do your outcomes plane inputs affect your process? Easy. Say you need to validate a major feature piece. You’ve been given 6 months to prove it’ll deliver the impact and you’ve now understood what’s realistically achievable.

So let’s start using Sprints because that’s everyone uses in the company…

Huh… Please don’t. Or do. But think about it.

You have a time box, you understand your constraints, what has worked and what hasn’t, what your team is and what the success metrics are going to be. Now is there a way to cheaply test the assumption? Can you spend 2 or 3 weeks rapidly iterating on prototypes? Can you go from no-set-process to Kanban, from no-estimates to t-shirt sizes? I believe you can, and you should, depending on the degree of risk and uncertainty associated with what you’re delivering. If the basics of your assumption are wrong, use the process to your advantage. Once the road ahead of you becomes clearer, re-evaluate and do it again.

3) Outputs — Roadmap

A backlog, a set of milestones, a way of working. Operational principles. The ceremonies. The social days. The deadlines.

All of these are designed to let your team advance towards the finish line and deliver something. We are only human and like to be rewarded, recognised and appreciated. Delivery is a reward in itself, but only impact makes it long-lasting.

4) Activities

There are many product ownership books and courses out there focussed on process. How to groom your backlog, how to estimate your feature cards, how to set deadlines, how to run reviews and retrospectives. How to identify the constraint and guarantee flow. There’s no need describe at length what the plays available are. Make sure your process fits the team, make sure the team understands why the process is a means to an end. Make sure you remember that progress isn’t impact, but should make it easier for you to get there.

Conclusion

Some products and features require you to manage more in some planes than others. The need to focus on each of them will also shift over time, but ultimately I argue a great Product Manager is strong in each of these 4 planes and is able to confidently move between them. Thinking about it in these terms helps me frame my job and the skills I need to work on.

If you’ve made it this far, hopefully it will help you too.