A Practical Setapp Guide For Measuring Product-Market Fit

If you Google anything around "product - market fit", you're going to find hundreds of articles saying why this is so important for startups and software companies. Without it, the experts claim, you can't scale your idea. Won't have a viable company. But there isn't much on how to implement this theory.

We put together this article to provide those in software marketing roles a practical guide for how to create and implement a product - market fit survey. And crucially, how to dive into the numbers, then what to do with the information you get from customers/users.

Product - market fit doesn't happen overnight. Although a lot of experts claim that product - market fit needs to be figured out before launching a new product or feature, many entrepreneurs and companies iterate with a live product, and real customers. Figuring out along the way. That works for a while, but every company gets to a point when it needs to get a clear idea what it is doing right and what needs improving.

To further refine our product and marketing, we used the product - market fit survey to find out what our customers think, and also, how we can improve.

How to run a Product - Market Fit Survey

Firstly, what is product - market fit?

Marc Andreessen, Silicon Valley entrepreneur, author, and founding partner of VC firm, Andreessen Horowitz (a16z) coined the term. It means:

"Product/market fit means being in a good market with a product that can satisfy that market."

Basically, if you want to grow, you need to know customers want your product and, ideally, can't live without it. And the market is large enough to sustain your growth: that enough potential customers exist and enough of them are going to love the product that growth can and will sustain itself.

For companies that didn't manage to do enough research into product - market fit before launching, you can use the Sean Ellis approach. Sean Ellis is a veteran startup marketing professional, responsible for marketing LogMeIn and Uproar from launch to IPO. He also worked with Dropbox, Eventbrite and Songkick, amongst others, and is now a consultant and Founder/CEO of GrowthHackers.

From his experiences, Ellis created a product/market fit survey. Each of the questions in the survey was designed by Sean Ellis to understand what users think about your product, what is their relationship with your product. We have applied this to our customer base to understand what we are doing right, to make sure we have product - market fit, and see where we can improve.

Here are the questions we used:

8 Simple Questions:

How to use this information?

Collecting survey data is always useful. We use this sort of information on a regular basis, to check we are on the right track with product updates, new releases and new features. But with this data, we - and any company that does something similar - is asking questions fundamental to why and how people use a product. It takes on new significance.

At the core of this survey is the question: "How would you feel if could no longer use [product]?"

If 40% of those who answer say "very disappointed", then you have achieved product - market fit. Imagine if Google didn't exist? Or Slack? Or Spotify? You want a response somewhere in the ballpark of: "I'd hate to have to figure out how to get what I need without this product!"

However, on its own, finding the answer to one question isn't going to change your business. You wouldn't be able to ask that question if you didn't have enough customers, so on some level, your product is viable. The real value is in the context surrounding that question, and the extra information you can gain by asking a few more questions.

One question, always worth asking, according to Hiten Shah, founder of KISSmetrics is "What would you likely use as an alternative", if your product didn't exist?

As Shah said in an interview: "It helps you understand how you fit against competitors, or what space you fit into. You might think that you're a customer feedback tool, while customers might weigh you as a customer support tool. So that question is actually pretty telling in helping you understand, from your customer's perspective, what other products they would compare your product with."

All of the information you get from this survey is well worth discovering.

When you are putting together a survey, here are the steps to take:

• Send it to those who will give you valuable responses.To get the best, most relevant results, send it to those who've experienced the core of your product offering, and have used it in the last two weeks.

• Send it to a minimum of 100 customers (the more, the merrier);

• Create your questions and input them into a survey tool - something you can use to send the questions and capture the information (we used SurveyMonkey);

• Know what you are going to do with the data, after you get responses.

What we have learned from Setapp:

We sent out a survey to 9000 paying customers and got over 1600 responses. It helped us discover how our customers (a nicer word that 'users'; and 'members' is even better) really feel about Setapp and what we have to improve.

We used SurveyMonkey to collect the responses.

The good news, we found from this survey is that:

Setapp Does Have Product/Market Fit

The key question in the product/market fit survey is the one that asks "How would you feel if you could no longer use Setapp?" The options for respondents were:

• Very disappointed

• Somewhat disappointed

• Not disappointed (It really isn't that useful)

• N/A - I no longer use Setapp

48% of people said that they would be "very disappointed" if Setapp no longer existed. According to Sean Ellis, if at least 40% of survey respondents make this choice, you have achieved product/market fit.

An Interesting comparison: Hiten Shah ran a product/market fit survey for Slack users, and here is what they found out:

With Slack's massive customer-based and brand awareness, it's no surprise that it has successful product/market fit. But it's cool to see that our numbers are even better when "somewhat disappointed" and "not disappointed" are compared.

Diving Deeper Into The Numbers

And here comes the segmentation. Every next question has to be analyzed by the choice users made in the key question: "How would you feel if you could no longer use Setapp?"

The segmentation is aimed to show the difference between users who consider Setapp a must-have (very disappointed respondents), users who see value but aren't there yet (somewhat disappointed respondents) and people who don't find Setapp useful (not disappointed respondents). Which helps get a deeper understanding of what makes people love the product.

Action Item: These conclusions have to be used to take action within Setapp on a product and product marketing level with one single goal: Get more customers that love Setapp.

Understanding Loyal Customers

Loyal and happy customers use Setapp as their main source of high-quality apps that they need. Here is what the survey results found:

• Setapp gives customers easy access to great apps they do not use often, but need to use on occasion.

• For them, it's convenient to always have access to a wide range of useful apps for daily needs, both personal and business.

• Setapp introduces them to a variety of really cool apps they did not know exist. And they have cool options to choose.

• With Setapp, customers can explore and discover a useful range of powerful apps to enhance their productivity, without them taking hours to find one. This has made Setapp a sticky product for them.

• Survey respondents say that Setapp helps them stay organized and saves a lot of time.

• These users like the fact that they are able to try out different apps without having to pay for each one separately, or try for a while before buying.

• Since apps upgrade automatically, and with so many choices, customers rarely need to use Apple's App Store anymore.

• Pricing: incredibly decent/fair/low price - according to survey responses.

Words and phrases people used to describe the benefits of Setapp included the following: Productivity; Easy; Flexibility; Convenient, Convenience; Time Saving; Ease of Use; Manage in one place; Single entry point; Access to apps; Always current version of app; Automatic upgrades; No headaches; Simplicity.

What is very interesting is that:

Respondents used emotive/negative language when talking about whether there are alternatives to Setapp. Although they can name the alternatives, (e.g. buying apps separately or in different bundles, finding free alternatives to apps available on Setapp), they sounded unhappy when thinking of the alternatives. This differentiates them from "somewhat disappointed users", who don't mind buying several apps they need separately.

Some of the responses included:

• Is there an alternative? Give me a break!

• Mac App Store :-(

• Nothing. there is no alternative!

• I don't know about alternatives!

• If there was such a thing. Your making me worried with your questions!

• I'm not aware of one, but I would look hard.

• Buy the apps individually (like before my Setapp days) but only the "must-have" apps :-(

• Buy selected applications which I feel are most beneficial to me, and sadly lose a number of others which are not vital to my work.

80% of these user recommended Setapp to someone else.

While describing Setapp to others, they were mostly referring to:

• Apps, their quantity and quality, and naming some on the list they have;

• Price;

• Ease of use;

• Comparing Setapp to Spotify and Netflix;

• Used lots of complimentary language: awesome, cool, fantastic, love.

What about customers who aren't as passionate about a product?

In many ways, the differences weren't great between very disappointed respondents (those who are most likely to recommend Setapp to others and love the product) and somewhat disappointed respondents, but we did find some when survey results came in.

1. They are not as concerned when considering an alternative to Setapp. Even if it does not exist, they will just go and buy some of the apps they need.
2. When defining the benefits of Setapp, they refer to quality, quantity, value for money, but they aren't as likely to mention the benefits of discovering new apps and productivity. Which tells us that these guys like the apps we have, the price, but they are not as interested in searching for new apps, beyond those they need to use. 

Open-ended responses

Open-ended responses are a key part of the product/market fit survey because they represent the thoughts of the people who actually use the product. We will use the words and phrases customers use to improve our marketing copy and increase conversions.

Why people chose their answer to "How would you feel if you could no longer use Setapp?"

What is the primary benefit that you have received from Setapp?

What type of person do you think would benefit most from Setapp?

How can we improve Setapp to better meet your needs?

From our survey, we found we have product - market fit. We were relieved to have found that, but also knew we were doing something right to have achieved our growth so far. It is one of those questions that is either best asked before launching, or when you've got sufficient traction to be sure you are on the right track already.

For those wanting to do the same, we've found the following approach useful for getting a better understanding of product - market fit:

• Create a questionnaire. Use the Sean Ellis method, making sure to include the key, "what if you couldn't use?" product question in this;

• Know who to sample - decide your criteria based on which group of customers/users has the biggest impact on the business, It is suggested to send it to those who've experienced the core of your product offering, and have used it in the last two weeks.

• Use survey software - something easy-to-use, for those answering questions and for collecting qualitative and quantitative data.

• Analyze the responses and make a plan for implementing the results.