Is the Pinterest Way to Measure Ads the Right Way?

Too Long; Didn't Read

A cutting-edge data science model can only be created if impact is measured properly. Pinterest upgraded everyone's preferred impact measurement metric, CTR, after carefully analyzing the problems. Every Data person adores the CTR measure as it is: Simple to explain to leadership, product managers do not question it, and measuring it is straightforward: Divide the number of clicks by the total number of impressions. CTR data is easily available, so this will be easy to calculate, track, and analyze. And, as a bonus, lots of advertisers want to have high CTRs too. So you’ve aligned your advertiser and user interests! So, just throw a lot of advanced machine learning at the problem to predict and maximize CTRs and you’re done, right? Not so fast! CTR suffers from some serious shortcomings. “CTR is not the key to user engagement.” CTR just reports on clicks on a certain item, regardless of whether it is relevant or not. Some may claim that the user clicked because the content or ad was relevant, however, this is not always the case; it could be due to positional bias, clickbait, or other factors. These flaws will be discussed in further depth below. CTR is only useful for measuring short-term gains without consideration of user engagement. As a result, we require a metric to track user engagement. “You can fool all the people some of the time and some of the people all the time but you cannot fool all the people all the time” - Abraham Lincoln