I conducted a lot of interesting and effective experiments. Some of them increased our metrics significantly, some of them less. I left the company several days ago, but I want to share something interesting. Today I will tell you how I managed to increase the user engagement of VK Messenger by adding a block of recommendations of contacts. And what is the most important – I will honestly tell you about the problems I faced and how I solved them because I think that is the most interesting part of stories like this one. The block of recommendations for contacts is located at the top of the chat list in VK Messenger. Here are the steps I took to integrate this feature: First of all, I studied the existing messengers and saw similar features in the products of our competitors like Facebook and Instagram, but I wasn't sure if it would fit well with VK Messenger. I had a hypothesis that it could increase the engagement of our users, so I decided to give it a try. In the beginning, I launched this feature based on a heuristical algorithm: people whom you added as friends recently, phone contacts, those with whom you chatted earlier, etc. I would like to conduct a RAT (riskiest assumptions test) in the cheapest possible way to make sure I don't spend a lot of resources on things I am not sure would give us meaningful results. While I was sure that this MVP was really viable, I invited the machine learning team to help us improve the recommendations. Eventually, when I launched this feature based on an ML algorithm on the Web version, I got a +1% increase in time spent and +0.72% in the number of sent messages. I launched it on our Android and iOS mobile applications. Now, here are the problems I faced in developing and launching this experiment: Our team uses the messenger for day-to-day work, and they were concerned if this block with recommended contacts would distract the users from normal communication in existing chats. I collected feedback from our colleagues and did user interviews to improve the user experience. As a result, I decreased the size of the block a little bit and added a toggle to turn off these recommendations for those who don't like them. Also, it's possible to hide certain people from the list if you like the recommendations but don't want to see some of them. Later, when I launched an experiment on our real users I recognized that this opinion was biased because our core audience consisted of young scholars and students, and they found it entertaining to see new people to chat with. I have different types of users, so the switching off feature wasn't distracting for our core users, but also it was well-received by those who wanted to see just their chat list without recommendations. When I launched this experiment on the Android version I saw a slight initial decrease in time spent on the Stories feature, but the total time spent on the platform increased. This happened because the number of users who spent their time chatting with recommended contacts grew significantly. So I decided that this feature should be launched for 100% of our users. Later, I would improve the metrics of Stories by adding the Stories feature to the Messenger Tab, making it a win-win for both of us. When I launched this experiment on iOS, our support team came to us with complaints that the users don't understand what it is and what type of people they see in this block. In just one night our developer created a chatbot that explained the algorithm and asked for feedback so that I could make it even better. I launched it the next day, placing it in the first position of the block with recommended contacts. To sum it up, let me point out the most important parts of the development process: It's important to launch the feature within your team first – eat your own dog food! Collect the problems and ideas on how to improve them, then try to solve these problems. Be careful about biased opinions, though. Conduct as many user interviews as possible to ensure you don't miss any important moments. Launch only on the part of the users first, get feedback, and fix the problems fast. Then launch it step by step, gradually increasing the percentage of the users who see this feature. Always try to find the most effective solutions that need less effort. It will help you move forward faster. Check that you don't screw up other parts of the whole service. And if you see that something is going wrong, find the correct way to calculate profits and losses to make the right decision. Be brave! Changing the parts of your service that weren't changed for years may seem risky at times, but being bold and fast is the only way to create a big impact.
