How to Improve Marketability of a Mobile Game Application with Concept Testing

Game publishers may have no idea that they have weak marketability until they soft launch their product, and at this point they have already invested huge resources into the development. Yet, there’s a way to make sure that the game can hit its business goals before writing a single line of code. I’m referring to game concept testing. 

Concept testing happens on emulated pages of the App Store and Google Play. Basically, you show the product page of a game that you intend to develop to your target audience to see how many of them click the “Get” button. 

The numeric results of such tests, or metrics, provide insights into whether your app has a good chance of success whatsoever. Overall, game concept testing is a data-driven approach to launching games and a viable instrument to continuously improve your marketing message.

Now let’s dig deeper into…

Why use game concept testing? 

The best thing about game concept testing is that it saves you a good deal of time, money and other resources for the development. On the pre-launch stage, you get data-driven insights into how the audience would respond to your game in total or to the new feature.  

You can start with A/B testing the game subcategory to answer the question “Should I develop a card game or a puzzle game?” This is the most general thing you can A/B test. 

A/B testing subcategories: card game vs puzzle game

Additionally, game publishers can compare a few mobile game concepts that they are considering and select the most well-performing one. You can also choose between a few positioning strategies or understand the most well-performing audience, like should it be a game for kids or adults, etc. 

A/B testing game positioning: quiz on brand guessing vs quiz on movie guessing 

Details can be validated too. For example, you can check which of the game features resonates better with your audience and once you understand them, get down to the development. 

A/B testing game features

Key game concept testing applications

Overall, there are two major applications for game concept testing. 

1. Theme testing. 

Theme testing can be done early on, at the prototyping stage of the game development when you are figuring out what type of game you should be creating. 

Here, you A/B test the core gameplay, or setting, as well as the elements in it. For example, the theme can be Middle Ages or Fantasy, and the elements can be dragons or zombies, etc. 

A/B testing two settings: space vs water world

Tests help game publishers see what users are more interested in, as well as find a less resonating theme, which is an equally important insight. 

2. Increasing IPM, or installs per mille

When you manage to increase the IPM (installs per thousand of ad impressions), you can lower your CPI. In other words, the better your ad and the app store page performs, the cheaper it is to acquire users.  

By experimenting with different styles and app store optimization and tying the creative to the gameplay, you can arrive at something that might resonate better than your current elements, and thus impact your IPM in a positive way. 

How to perform game concept testing

1. Form a hypothesis.

To build one, start with analyzing the market and check if there are similar games out there, whether they are in demand and what features they have. For the A/B tests to have really meaningful results, the tested concepts should differ distinctively, not just by the color of the button on the third screenshot. 

At this stage, make sure you know your audience. One of the mistakes that some game publishers make using the SplitMetrics platform is A/B testing on a very small audience. For example, instead of targeting the United States or several countries in Europe, they run A/B tests on, let’s say, Switzerland. As a result, they get a very low conversion and, moreover, the conclusions based on such are far from being meaningful.  

Another mistake related to the audience is choosing the wrong age. We saw some publishers failing their game concept testing for a younger audience, while they could launch the same concept for an older segment of users and that would work. The only recommendation to give here is to study and know your audience.

After you’re done with the hypothesis, you have to decide what your game should look like.

2. Create a pretotype of your game. 

That’s right, a pretotype that comes before the prototype. 

Let’s say you’re hesitating between the two settings – a cybercity and an alien planet. To understand the best direction cost-efficiently, you could purchase the two packages from an asset store. Then you may want to modify them, put on screenshots and run tests to see which one is more resonating with your audience. 

Thus, instead of investing time and money in creating your assets from scratch, you just buy them for a hundred dollars and get numerical user feedback behind each of the environments under A/B testing. 

3. Build creatives for an emulated app store page.

To do that, you have to design the app store visuals based on the pretotype of your game. These are a few versions of the product page with elements – an icon, screenshots, videos (potentially) and UA banners.  

Next, you create a copy, or text variations for the app – a title, subtitle and app store description.

Once you’ve got the visuals and copy prepared, you’re all set for showing the app page variations to potential users.

4. Run A/B tests of your game concepts. 

There are two ways to do that using SplitMetrics. 

Firstly, you can run a one-variation experiment to check the overall performance and conversion rate. This type of test will give you the right direction in a very cost-effective way: you don’t need to waste money buying the traffic to test several variations while you’re not sure whether your overall concept will work or not.  

Secondly, you can perform multivariate A/B tests, where one variation stands for one concept, and check which one would perform better.  

After you’ve found the top-performing variation, build a strategy to improve it until you’re happy with your results and ready for the launch.

5. Analyze metrics.

To assess the performance of variations under A/B testing, you have to consider their conversion rate, time on page, bounce rate, engagement rate, direct installs against the screenshot view, and the screenshot full-screen view, which basically means an open rate. 

At the highest level, look at the IPM, or installs per thousand impressions. When assessing the metric, you may benchmark your own IPM against the game genre standard, but most importantly, compare it against your past performance. The IPM will change either way as you’re making changes to your assets.

How to improve the app store conversion rate? 

Improving CVR will directly increase the install rate of your game and, most importantly, optimize the cost of acquiring users. So what are the ways to achieve the app store conversion rate improvement? 

The major CVR improvements come from optimizing screenshots and previews of your game. However, our findings show that two and more videos may decrease the conversion rate, so you might want to check the hypothesis that you could perform better on the app stores with no videos whatsoever.

You can improve the conversion rate by adding the brand name to your game’s title on the App Store or Google Play. However, this might work for big brands in the first place. 

Take into account seasonality: adding two snowflakes to your icon or screenshots before the Christmas holidays might make a big difference. Just test if it works for your game.

Another thing that you can test is localization: find out if you need to adjust your app page to a particular storefront, and that may give you a major boost in another region. 

Final thought

Concept testing is not a one-time event, but a continuous process of improving your game. It’s really important to keep running tests at the prototyping, pre-alpha and alpha development stages to make sure that the marketability suits the core loop of your game.

This article is based on the podcast organized by the Deconstructor of Fun. Listen to us sharing even more valuable insights on how to improve mobile game marketability.

Previously published at https://splitmetrics.com/blog/game-concept-testing/