Why, When And How to Run User Interviews

Customer, or User interviews are invaluable for product and SaaS marketers, providing you go about this the right way. Here at MacPaw, we love data. If we don't measure it, we don't keep investing in something. Data has guided a lot of our decisions since before and after launching Setapp.

Some data points, however, aren't quantitative. We can’t always fit them neatly into graphs. In some cases, if you’re following Lean principles, part of the learning process means talking to users, seeing their experience through their eyes, instead of through dozens of intricate data points and analytics tools. We've learned a lot from this experience, so I would like to share with you some of our key findings and hope they prove useful.

In this article, we will cover the following:

1. Why you need to run user interviews?
2. How to invite someone and who should you invite?
3. What to ask and not ask?
4. How to make use of the collected insights, data and feedback?

Conducting User Interviews

At the heart of every startup should be a fundamental essential that they are, in some way, solving customer problems. Creating something of sufficient value that customers are happy to pay money to have that problem solved. Without user interviews, how will you know you are achieving these core aims?

#1: Why you need to run user interviews?

With that in mind, you could be forgiven for thinking that user interviews are all about giving your team feedback, or a well-deserved pat on the back. Neither is really the case. Yes, if a customer says “Well done, keep doing what you’re doing!”, that’s wonderful, and you should ask if that could be used as an online testimonial. Every positive review helps.

At the same time, if someone says they’ve had a terrible user or customer experience and they welcome the chance to complain, this is equally useful. Startups need feedback. Good or bad, every single one is a learning experience.

But beyond positive and negative feedback, user interviews are one of the best, if not the best way to get the information you need to shape product, marketing and customer service questions. If you aren't doing these yet - START NOW! We can not recommend user interviews highly enough. One reason these are useful is the results can indicate what product features to build quickly and what to keep further down the pipeline.

Despite a lot of marketers knowing user interviews are useful, one reason we know they fail is not everyone asks the right questions at the right time, and marketers don't always talk to the right audience. After conducting hundreds of interviews over the previous ten months, we’ve now got a pretty good idea what to do/not do.

#2: How to invite someone and who should you invite?

Something we learned very quickly: Don't offer an incentive for the interview. All you end up with is data that isn’t really helpful, from customers who aren't big advocates, who only wanted whatever you were offering.

Before and after launching Setapp, we were keen to talk to as many users as possible. We offered incentives, including the chance to win one of five Apple TV’s. Pretty soon, we were overwhelmed with people tripping over themselves to sit down for an interview. The time in my diary I used to leave free for lunch soon disappeared.

We were delighted with the response. A lot of the feedback was positive, which was great - it re-enforced our assumptions that we were on the right track and giving customers exactly what they wanted.

Brilliant! Everyone could relax and finish early.

Except, not really. Voices of reason chirped up in the office, and alarm bells soon went off. We checked most of the user accounts, to see who was actually using the apps on a regular basis and were surprised to see that most of our “passionate brand advocates” hardly ever used the product. The majority weren't even paying customers; they had only created accounts then done nothing, not even downloaded the product.

All they wanted was the chance to win an Apple TV. Even if we’d made the incentive smaller, such as an Amazon gift voucher, the result would have been similar. We learned the following:

• Don’t offer an incentive. Genuine users are always going to be willing to talk to a company without a prize;
• Invite the right people. Active customers, those who will have something to talk about;
• Clearly define, for your product, an active user. Make sure they’ve got beyond the download screen.

#3: What to ask and not ask?

Asking the right questions (to the right customers/users) is essential. Unless you get this right, the whole process will be a waste of time. You need to create a questionnaire, print it, stick to it - or change it if you’re finding that some of the questions don’t, on reflection, make sense.

Firstly, know what you want answered, what questions need to be asked. But don't ask questions that lead customers to pre-determined answers. Leading question isn't allowed in a court of law, and they're not useful when conducting user interviews.

For example, “Why would you prefer to pay annually, not monthly?” is a question we asked. It could have been worded differently, skewing it to sell the benefits of annual instead of monthly payments (e.g. get a discount), but we needed to know what other reasons customers would give for going with an annual plan.

Some of the answers included that it’s easier to pay annually, no need to worry about another monthly payment, no need to update card details if they’re changed/lost or stolen. Customer's reasons for doing something and company's aren't always the same. User interviews shine a light on customer's reasons, which is how you design better products and experiences.

Here are a few other vital things we learned when designing a questionnaire:

Structure your questions logically

This isn’t a police interview on a crime drama. You aren't trying to catch them out. You need sensible answers you can use, so create a structure - with a beginning, middle, end, and the chance for them to ask questions too.

Ask open-ended questions

This isn’t about more data metrics, this is about getting real insights from real customers, and you can only do that when you give them a chance to talk about experiences that relate to your product/service.

For example, “What apps have you bought recently?” is completely different from “How do you find good apps?” The second gave us loads of insight, something that even analytics tools can’t provide.

Asking the right questions will give your marketing and product the teams the chance to sift for golden insights that could make a huge impact on product development.

Never ask, “Would you buy [insert product name]?” 

Users will almost always say “Yes.” They don't want to look silly, especially if they've been invited by the company selling the product. As humans, we feel obligated to say yes, even if it’s not true.

Build rapport

It sounds obvious, but this interview is another interaction point between a brand and the customer. Show them the human face of the company and make sure they leave with a good impression. Also, don't be afraid to ask for a positive testimonial or something you can use on social media, providing the interview has gone well.

#4: How to make use of the collected insights, data and feedback?

Of course, this information is so valuable in the product roadmap.

But that isn’t the only part of the business that could find these insights useful. We compile the information and share with other operational areas: Marketing, Business Development, Account Managers, Customer Support, Project Managers, and Stakeholders.

We also print this information, making sure everyone can see who we are creating products for, putting a User of The Week Profile in a high foot traffic area of the office.

Why User Interviews Fail: Key Takeaways

User interviews are one of the most useful ways we have to create apps used by people across the world. Make sure you ask the right people. Don't offer incentives - they're a waste of time.

Ask the right questions - open, not leading, giving you the chance to have questions answered you need to know and uncover unexpected nuggets of gold insight.

User interviews are real insights, from real people, shaping how we build and improve our product and apps.