How to Test a Product Concept for $1,000

What does it take to turn a wild product idea into a roaring success? Let us introduce you to the journey of Magnet — an innovative app from the venture portfolio of Social Discovery Group, that invests in social discovery startups across the globe. Its main idea is to use cartoon avatars instead of real photos: people can’t see each other’s photos until they understand they both vibe. This certainly brings the individual features of a person to the fore.

Before the app launch, the team needed to test some assumptions and, most importantly, to prove that the concept was in demand. To find this out, the team decided to perform usability testing in the form of a clickable prototype. They nailed the product idea with only $1000 and showed it was a real hit. Curious how they pulled it off? Dive into the full article for the details!

What were the objectives?

The purposes depend on the specific study, but they generally include the following:

• Identification of design restraints

Barriers that prevent users from achieving targeted actions.

• Detectıon of growth points

Anything that helps make the design better and stronger.

• Learning about user behavior and preferences

Based on your target audience.

Major challenges

Cooperation with a third-party company can be really costly: the minimum entry amount can be about $5,000. Testing among your own community may not reveal the real picture, as your friends or colleagues might simply not be the target audience of the product. Taking this into consideration, we decided to utilize an online platform where respondents can be chosen from different countries.

Online-platforms

We reviewed two options:

• Userbrain

First and foremost, it involves reviewing videos of those who have used your product. Essentially, these are video recordings based on a scenario you specify. The link can lead to either the actual product or a clickable prototype.

• UsabilityHub

This is a more versatile platform. Here, you'll find 5-second tests, surveys, viewing heatmaps, and first-click tests. Also, you can also integrate a clickable prototype from Figma (this feature has become available relatively recently). Since we intended to conduct not only qualitative but also quantitative research, we decided to go with UsabilityHub.

How many respondents do you need?

In 1993, web design guru Jakob Nielsen said, "Five people are enough for testing." From the article by Jacob Nielsen, it follows that a sample of 5 respondents is enough to identify 85% of issues on the product. The curve chart clearly shows that you need to test at least 15 users to detect 100% of usability issues in the design area. Indeed, it is worth mentioning that this works only in the case of quality testing, and for quantitative testing we need at least 20 people. 50 people took part in our test: 25 men and 25 women from the USA, at the age of 25 up to 50 years old, with an income of $40,000 per year.

Our goals

1. Identıfy weak points in the application
2. Evaluate the entire positioning idea

Scenarios

To perform the test, our respondents had to do the following:

• Complete the registration

To identify the drawbacks the user can face throughout the entire regıstratıon process.

Check out the main sections of the app

Does the user understand where he is inside the app and how he navigates within the application.

• Try chatting with a user

Here comes along the main path leading to the first form of payment.

• Answer the key questions

Getting feedback, after interaction with the prototype.

Let’s dive deeper into each of them:

Complete the registration

1. Which color did you pick for your avatar's background?

   Many users have chosen blue as the background.

   This is most likely due to the fact that blue is set by default first. Based on this, we plan to add text, as a simple and simple call to action - "Choose the background."

   
   

2. Would you be interested in finishing the sentences or would you rather skip them?

   72% of men and 52% of women have chosen to complete all phrases.

   Considering that the female audience had split, we decided to make mandatory fillıng out only one phrase, and the rest can be missed.

3. Could you explain how our app works after reading the tutorial?

   More than half of all respondents described the mechanism correctly.

   Given the mixed result, we plan to make the guidance in the form of tips after registration.

Check out the main sections of the app

1. How many profiles did you see?

   More than half of all respondents did not understand that profiles could be swiped.

   Post-registration clues could also help with this matter.

2. Which one of the provided profiles would you like to chat with and why?

   While the majority of the male audience paid attention to interests, the female respondents focused on clue phrases.

   This asserts the importance of separating the test for male and female audiences. The results can be substantially different.

   
   

3. What could make it easier to choose a profile to chat with?

   Respondents answered that more info would allow them to navigate better. As we noticed on the heat map, they clicked on the avatar, waiting for an action.

   This prompted us to the idea of ​creating a full profile, with the opportunity to click on the avatar.

   
   

Try chatting with a user

Would you prefer to continue chatting without revealing your real photos?

Only 28% of men and 20% of women want to continue communicating with a hidden photo.

We looked at what motivates them to proceed with this format. It turned out that it gave them a sense of security and safety. Also, they didn’t feel tied up to a person's real appearance. Such responses were in the minority, but we thought about creating anonymous chats.

Answer the key questions

1. Tomorrow, you will have 5 new profiles to chat with. Would you come back and continue using our app?

   88% of men and 80% of women gave a positive response.

2. Did you like the concept of the app?

   92% of men and 84% of women gave a positive response.

3. How close would you say our app is to a dating game on a 5-point scale?

   Evaluatıon coefficient by men- 3.6 from 5, by women – 3.9 from 5.

Conclusion

Testing of ideas and assumptions at the early stage of the application development allowed us to get feedback from real users, as well as save development resources - we didn’t have to develop the feature set in order to test it. It is also worth mentioning the cost of our study, which amounted to only $1000. All this made it possible to identify issues that were not visible before and update the roadmap of the product. The outcomes from this testing gave us tangible business impacts:

• Registration optimization

Increase of the percentage of users who have reached the end of registration.

• Revision of the manual

More effective user engagement through special tips.

• Full-featured profile

Additional motivation to explore profiles and stay in the app.

• Anonymous chats

Meeting the needs of users, increasing the average check due to new functionality.

Inviting users into the heart of product development is an essential stride toward success. Our journey shows that concept testing doesn't necessarily involve substantial expenses. We're sticking to this approach, always asking users what they think when we're adding new features to Magnet. With their feedback guiding us, we truly change the way people communicate online.

Written by Alexander Belski, Lead Product Designer, Magnet