“White supremacist”, “misogynist”, and “Putin’s spy” are just a few of the names I was called in the US during a week of hype surrounding our MakeApp project. Was I surprised? Yes, I was. Here’s why.
We are a team of just 10 developers and graphic designers. Our flagship product is the AI-based augmented reality app called Magic. It uses emotion recognition for games and 3D masks.
It grabbed Apple’s attention and was featured on the App Store time and again.
Magic on the AppStore
With Magic, we were developing a photo-realistic digital makeup technology.
While experimenting with it, we found out that neural nets can both apply and remove makeup.
“Hmm, that might be interesting,” I thought and suggested that we tap the market using this feature.
All our Russian team members voted in favor of launching this experiment. Only one team member, Pir, my close friend and a US citizen, was against it.
He argued that the US media would dismiss this feature and accuse us of sexism. Having lived my whole life in Russia, I could hardly understand what he was talking about. I could not realize what was wrong with this makeup removal function. I listened to all the arguments and decided to keep this feature and release the app.
The first MakeApp version appeared in the App Store six months ago. We posted it on Product Hunt and caught the eye of Ryan Hoover, its founder.
“Oh, that’s great,” we thought and went to bed.
When I woke up the next morning to check my email, I could hardly believe my eyes. My inbox was bombarded with dozens of emails from journalists from Japan, South Korea, and China. They all asked me for an interview and comments about MakeApp.
I went to the kitchen and made myself some coffee. I understood what was going on after I looked at the stats. We blew up the Asian market, went viral and topped the App Store ratings.
AppStore in South Korea
MakeApp became Asia’s top story. Look, Japan’s TV presenters are loving it!
Abema TV
Asian media were positive about our project, and Twitter users were really excited too. My American friend’s concerns have proved unfounded. “Asia loves us!” — he messaged me.
Media in China, Japan, South Korea, and Thailand were teeming with very favorable headlines. MakeApp became a bestseller in just one day.
The project was so celebrated in Asia that China’s Tencent announced that they would develop a competing product.
“Cool! Media and users love us! We are on the right track,” our team cheered and continued to pursue this technology.
Half a year later, we added video processing to MakeApp. I thought it would be great to post the news on BoredPanda. I decided to take celebrities’ videos and pictures as sample images. Their looks are well-known and users can easily recognize them without makeup to see how the technology works.
I prepared photos and GIFs, published them, and shared the links in relevant subreddits.
BoredPanda editors saw the material and shared it on their Facebook pages. We were amazed to see Europe explode this time. MakeApp took the UK by storm, followed by Germany, Sweden, Italy, and others. The Daily Mail repeated my experiment and tested out the app on stars. MakeApp hit the news headlines.
The Daily Mail’s main page
Our happiness knew no bounds! Everyone was thrilled! Everything went great.
Well, it was — until the US media took notice.
“A male Russian propagandist is behind an unflattering AI app that shows how women look without makeup” was the Business Insider headline that marked the start of our story in the USA.
I was disappointed to find out how unprofessional journalists are in these and some other “quality” US media outlets.
My correspondence with Shona Ghosh showed that her primary interest was to hype the clickbait story rather than to investigate the app and technology. “Russian-Kremlin-spy-sexist-racist” — heck, this guy is a dream come true for the yellow press!
From our correspondence, I became aware that her agenda was to make me into a wicked monster and the app into a spawn of hell. Which I mentioned to her.
Part of an email correspondence with Business Insider
Another funny thing. The reporter from Business Insider asked if we had women in our team. Unfortunately for the inciter, we have women on board, a young and talented girl who deals specifically with neural nets.
Part of an email correspondence with Business Insider
Do you think the story mentioned her? No, it did not. She undoubtedly asked this question to get my “no” and then write that the sexist app had a male-only team.
Business Insider tried to accuse us of racism. Shona said the app made skin whiter and tried to prove this by sending me Serena Williams’s pictures.
Sirena William’s image processed with MakeApp
I asked her where she saw “whiter skin” in the image but she did not bother to respond.
It was amusing to read a technology review written by a person who knows nothing about technology.
Especially given that our team has made a breakthrough in the field of face retouching. Here’s why.
Real-time digital makeup on smartphones is currently done using face tracking and projection mapping. The result will directly depend on whether the face can be found within the frame and how well its landmarks can be identified. When processing a large number of frames, the quality of tracking is especially important, as a glitch may result in the “loss” of makeup.
Because of underlying face detection and tracking steps processing multiple faces in a frame is a very challenging task for this method and existing solutions only limited to one face per image.
To illustrate my point, I’ll show you how this works in one of the most popular beauty editors, YouCam.
Made with YouCam
Capturing facial structure and projection mapping is also a problem. If the face is partially covered by another object, the present-day technology just ignores it and puts makeup over it.
Made with YouCam
Besides, we should not forget about challenging situations where the face is slightly out of frame or doesn’t fit into it — as is often the case with phone cameras.
So we developed a radically different technology where the image is processed entirely by a neural network, thus providing a partial or complete solution to all of these issues.
This is our technology at work.
Made with MakeApp
Our approach is based on the GAN (Generative Adversarial Neural Network) training algorithm, where the model attempts to imitate real data in order for the makeup to be as authentic as possible.
The resulting generative model takes into account many factors that are hard to capture programmatically: lighting, skin tone, photo quality, etc. As a result, the makeup is almost photo-realistic.
Unlike the tracking technology, this method makes it possible to remove the makeup by regressing the face image using the data learned in the process.
As far as we know, this is the first GAN-based real-time model for mobile devices and the first one to use a generative process for applying and removing makeup.
But the journalists could care less.
The only thing that seemed to interest them was the “Russian male–sexist–propagandist–who wants to make women ugly” narrative.
After the article had been published, the media exploded:
I must confess that I was bitterly disappointed. Trained as a journalist, I had been working in the Russian media for over 10 years. To me, the US media had always been a paragon of quality journalism. Imagine my surprise when I was contacted by representatives of just four newspapers other than Business Insider: Huffington Post, Buzzfeed, The Daily Dota, and Mashable — that’s it.
The rest of them not only failed to make contact, but they even ignored my own emails.
They generally reprinted the Business Insider article without so much as an attempt to do any fact-checking. Some went even further and resorted to blatant fabrication. Elle Australia wrote that I had developed FaceApp, an app that had been previously accused of racism. Guys, I have nothing to do with FaceApp!
Elle Australia
I tweeted Elle Australia’s Editor-in-Chief. Again, no reply.
In order to boost their traffic, journalists tried to present me as a Kremlin agent, and MakeApp its product. In need of proof, they visited my Wikipedia page which stated that I had created the Russian LifeNews channel. “Here’s the proof! MakeApp came right out of the Kremlin!” Right, because every Russian is a spy. At least that’s what the journalists seemed to think.
LifeNews is similar to FoxNews in the US and SkyNews in the UK. When working for the channel, I drew on the expertise of my American and British colleagues. Two years after launching, it had become the most popular TV news channel in Russia and received the World Summit Award for its contribution to citizen journalism.
LifeNews always prints exclusive and explosive stories about life in Russia.
Now, here we’ve got Andrei Arshavin, captain of Russia’s national football team, asking a waiter to bring him whiskey after being defeated during European Qualifiers.
Russia’s national soccer team wanted to sue LifeNews following the publication.
And here is the Russian mogul, Oleg Deripaska, eating black caviar with a spoon, a couple of weeks after the announcement of layoffs at his plant due to financial difficulties.
LifeNews has the strongest journalist team in Russia, and they always provide photo, video, and audio materials as proof. This is fact-driven journalism. To call it propagandistic is ridiculous at best.
As for myself, I used to be the channel’s CEO. As you know, a TV channel is usually divided into two units: editorial and technical. I was responsible for the channel’s technical equipment. I must confess that I’m proud of us for having built Europe’s biggest news room, together with a multi-use studio, in a matter of 7 months. You can see our creation here.
LifeNews’ office with studio and newsroom
I asked the journalists what was so propagandistic about me but they didn’t provide any answer.
Information provided by AppAnnie
But our American users stood up against the criticism. You can see most of them praising MakeApp.
Our team receives numerous messages of support from our American users.
Moreover, our project didn’t go unnoticed in the professional community. Here’s the founder of Oculus VR retweeting my messages about MakeApp.
I must admit that we owe a certain amount of our success to those unprofessional journalists. It was their sensationalist fluff that helped us acquire over 200,000 new installations and top the US App Store ratings over the course of five days of hype. Besides giving us their appreciation, our grateful users also support the project financially. We began to earn thousands of dollars per day.
Today we launched a new MakeApp version that allows real time makeup application and removal. iOS users (6S and higher) can download the app here, and the Android version will be launched a bit later.
MakeApp now works in real-time
We would appreciate it if you could support our project in any way, whether via monetary contributions, a positive attitude, or simply by sharing this post.
Your craving for the truth and willingness to speak up is our team’s biggest success.
Best regards,
Your Russian spy,
Ashot Gabrelyanov
MakeApp & Magic
Founder