Real-time Machine Learning For Recommendations and Search Engines

My name is Fedor Gvozdev, and I am the founder of Holyskin - an online store of Korean cosmetics. We have been on the market since 2013, and during this time, we have been able to test many machine learning algorithms to increase conversions and improve the display of personalized offers. In this article, I would like to talk about real-time machine learning for recommendation and search engines.

Machine learning in recommendation systems

As you all know, nowadays, any application or platform somehow uses machine learning to collect user data. In general, machine learning is the study of a computer without programming, using only templates and logical conclusions. Machine learning is successfully implemented in recommendation and search engines to increase website conversion and offer users relevant or interesting content and products. For example, in social networks, this is done to keep users on the platform more, to interest them in advertising integrations, and in e-commerce to offer customers relevant products and eventually sell them something.

Benefits of real-time machine learning

Personal recommendation systems improve the user experience, giving the user a good impression of your platform as a convenient service. It also helps to provide qualified guidance to customers thanks to less information load, offering precisely the product that the customer wants. In addition, the real-time machine allows you to make recommendations based on short-term user interests and real-time demand. This allows you to meet the buyer's needs and reduces the number of rejections. Also, real-time machine learning gathers valuable information for managers to improve logistics, supply, and marketing. Ultimately, this leads to an increase in revenue and conversion by an average of 6-18%.

A sorting algorithm for displaying personalized offers

There are many algorithms for sorting personal offers, but they all go through three main stages. First, these algorithms identify trends based on primary and secondary data, as a result of which hypotheses are built. Many such hypotheses are tested, for example, using AB testing or other types of testing. As a result, managers will identify the most optimal strategy.

Data collection

Data is collected based on user actions, and traffic can be divided into three types - cold traffic, warm traffic, and purchase traffic. Most cold traffic is people who don't know about your company or brand at all but have only seen it in ads or heard about it from someone else. Warm traffic is those people who are familiar with you and are at the stage of deciding on the form of a purchase. And, of course, the last type is those who have made purchases on your platform.

These people who come to your platform leave explicit and implicit data that you can easily collect and further analyze. This is where the data collection takes place. Explicit data can include reviews, ratings, search history, saved products, and so on, and implicit actions are product clicks, scrolling, and browsing. These actions help identify the data that is of most interest.

Data storage

The next step is data storage. For this, you can use a database of your choice. But the main criteria are adequacy - the relevance of the database to the real subject area; completeness - the ability to meet the user needs; adaptability to changing conditions. A scalable database will be easier to manage.

Data analysis

An in-depth analysis using previously obtained data is necessary for an accurate, high-quality recommendation or conclusion. At this point, it is worth talking about the timeliness of the analysis. For example, several types of analysis can process data in real-time or periodically every few minutes or seconds. But we are interested in training the system in real-time. Real-time analysis is capable of producing recommendations instantaneously. The advantage of this type of analysis is that there is less chance that the user will leave the site before finding the product they are interested in. Also, it increases the competitiveness of your platform.

Part of data analysis is data filtering, where inappropriate data is filtered out based on similar attributes, opinions of users with similar tastes, and similar products.

Recommendation output

Holyskin is an online cosmetics store with a list of more than 10,000 items. No matter how many thematic subdirectories were distributed in the catalog and filters were added, all users, without exception, found it difficult to choose as we expected. To simplify it, we implemented a real-time recommendation system. It allows users to see a list of products selected for the first in the catalog. This output is based on their and other visitors' search history. Also, it is based on the demand for the product on the site and in the media space. In addition, we added recommendations to the product pages by creating a block called "You may be interested". This block featured products from related categories that other visitors were interested in or even bought along with the product they were viewing.

It was crucial to add the concept of real-time directly since our list contains products of different focus, such as anti-aging skin care vs. eye makeup. Products from these categories can interest a wide range of users with various interests. For this reason, it is necessary to display immediately relevant recommendations for users.

This recommendation system keeps visitors from losing items they've viewed and offers them more relevant products. As a result, the company's conversion rate increased by 14%.

Examples of recommendation systems in well-known companies

Giant companies have long implemented a recommendation system on their platforms.

YouTube collects data on likes, dislikes, comments, subscriptions, views, time spent, reposts, and even geography, which helps to show more relevant content.

Linkedin has created an algorithm to select the appropriate candidates for a given position, making it much easier for recruiters to find them. Moreover, this tool is widespread among HR professionals.

In addition, Amazon, the largest e-commerce site, implemented the system in early 2010 and increased sales by 29 percent.

Netflix, for its part, uses not only users' interests and views but also the topics and genres of movies and their behavior patterns for more accurate recommendations.

Conclusion

Recommendation engines help meet user needs and simultaneously improve the productivity of companies and enterprises. Although there may be ethical issues such as user privacy and user rights violations, in most cases, the benefits of referral systems outweigh the drawbacks. So this area still needs to be improved and refined to ensure user privacy and personal data security.