Developing an AI mobile App: Our Experience, Mistakes, and Achievements

Every Product Manager wishes that their app will change the lives of its users for the better. This was the case for me too when we just started working on the AI mobile app CountThis. In the beginning, the app was supposed to instantly count similar objects in a photo with the help of our own neural network. At that point, we didn’t have a limited list of objects for counting; instead, we wanted to cover as many application spheres as possible. However, as we kept developing the app, we started to focus on certain categories, that is, on the accuracy of the result. The “less is more” rule came into play in this case.

It all started with a marketing creative

The story of CountThis (AI mobile app that quickly counts similar objects through the phone’s camera) started with a great marketing creative. Our colleagues from the Marketing Department are constantly testing various ideas. At some point, they came up with an idea of a creative about an app that counts items in a photo (a lot of logs, to be specific).

What followed was a clip that showed AI instantly counting completely different objects. It received a large positive response from social media. The team was convinced that the concept had potential, the Machine Learning specialists “tinkered” with the first version of the neural network, and implemented the counting mode in one of our apps to see how real users would react. This is pretty much the beginning of CountThis, and soon afterward, it became a separate app in the AIBY product portfolio.

Initially, we didn’t pick any specific category of objects to count; instead, we were trying to train the neural network to count any type of similar items in the picture based on the object selected by the user. Later, we had to switch to a different approach. We realized that putting quantity (a lot of different object categories in our case) over quality is not worth it. Basically, training the AI to count keyboard keys, concrete slabs, and pills are 3 different processes that are time and effort-consuming. Users expect the app to count quickly and accurately, which is perfectly reasonable. By spreading ourselves between several counting categories, we wouldn’t achieve high quality and accuracy of the result—the two criteria that became our main focus. That’s why we decided to introduce new counting categories gradually. At the moment, we are focusing on the spheres of construction and medicine.

The time it takes to train a neural network to count new objects depends on:

• whether you have a good category dataset, which simplifies the training process and reduces time;
• the “complexity” of the category. If it contains items that can be easily distinguished in a picture (e.g. boxes), everything goes smoothly. But if it’s something denser (e.g. sheets of iron or cardboard), the training gets more difficult and time-consuming.

On average, it may take anywhere from 2 weeks to a month to train AI to count objects of a certain category.

Dealing with challenges—how to choose relevant spheres for an app

This raises an obvious question: why do we focus on the construction and medical fields? The answer is quite simple. After several iterations of the product, we gained a better understanding of what our users were trying to count and what we were good at counting. We received a lot of feedback from construction, industrial, and pharmaceutical companies, where employees often have to count a large number of construction materials and pills.

At this stage of app development, we realized that counting various items equally accurately is impossible due to the current level of technology. That’s why we decided to focus on improving the counting accuracy only in certain spheres.

Obviously, we are not thinking about stopping here! We are planning to expand the list of the counting spheres beyond what we are working with right now. To get an idea of what item category we should introduce to the app next, we are looking to the following sources:

• Items are counted by our users. We collect the information about them with the users’ permission, which helps us learn what’s relevant for them.
• Clients. Usually, they want to count items of a specific category, but we can find out what else they need to count.
• User surveys. We find out our users’ needs through surveys and consider the results of creatives prepared by the Marketing Department as they continue to test various counting hypotheses.

As Product Manager, I do additional work to research new possible spheres for CountThis. This includes continuous research of the market, users, and competitors, generating and checking hypotheses (with as few resources and as little time spent as possible), understanding of the technology limits and how to get the most out of it. Dealing with data, interviewing users, and studying the domain are all great ways to accurately determine the most relevant items for counting and focus on them above all others.

Moreover, we are constantly improving the app’s UI. User interviews and analytics help us find out what’s lacking or working incorrectly, unintuitively, not how we imagined it, or is inconvenient to use. This is a great field for study and examination that reveals non-obvious scenarios of use. In a nutshell, all of them are areas for improvement. For example, we know what kind of conditions certain users take pictures of objects in. We see that quite often, there isn’t enough light or the smartphone camera is of poor quality. Later, we can try to solve these problems while training the neural network or pre or post-processing a photo to get the most out of the current technology.

Why is it so interesting despite being so complex?

Working on an AI app is a challenging but incredibly engaging experience. Why do I like being a Product Manager for a product like this one?

It’s a new project that allows you to go through all of its development stages: from an idea to a fully-fledged product that helps people. This work is at the forefront of computer vision and machine learning. It involves communication with the brightest minds who are turning technology that used to seem like science fiction into reality. A big part of this job is interacting with users, which gives you a sense that your product helps them solve their problems and motivates you to keep improving it.