This is the first article in the series on programmatic advertising. In this series, I will introduce the reader to the world of mathematical problems and machine learning challenges arising in the world of online advertising. We will start with a simple introduction and brief overview of the ecosystem and business models in general, slowly going into more and more mathematical detail. The first collection of articles will provide a short introduction to the programmatic advertising ecosystem and will only briefly mention arising challenges and mathematical problems that will be covered in greater detail in the later articles.
The main players
What happens when you open your favorite news website and see another shoe ad? Most probably, the ad was selected automatically through programmatic advertising, which we will refer to as Real-Time Bidding, or RTB. This means that multiple computers make complicated computations and exchange a lot of HTTP requests in real time just to select and show the ad that is thought of as the best for you right now.
Well, best for everyone, including you to some extent. Let’s look at the whole ecosystem and companies involved.
The user
The User is you. You open your usual news website to catch up with what is going on in the world. In a couple of places within the news article there happen to be some ads. Most of the time you ignore them, but once in a while ads actually promote something interesting for you, and you might click on them, or even buy something on the advertiser's website afterwards.
Supply-side: The Publisher
Your favorite news website runs a business model that is called Publisher. When you open the news page in the browser, your smartphone sends a request for a web page to the website. While figuring out what kind of news to show to you, the website is also finding the options of ads to display — and sends a reply to your smartphone with a formatted web page containing news and ads personalized just for you.
Usually, the server is expected to return a fully formatted web page with all the content and ads in 1 second. In order to choose the ad, within this 1 second the website usually does the following: groups information about:
-
the page you are about to see (URL, maybe some keywords and classification tags),
-
the ad space (size of the ad to be shown, its position within the page)
-
and about you (your browser, IP address, and anything else it is able to find)
Then it sends this information to the ad exchange, asking: “Which ad should I show of this size, on this page for this user?” — we will call this Bid Request or Ad Request. Ad exchange replies with the ad, which the website puts into the page and sends the page back to you.
Supply-Side: SSP
For the sake of this article, we will treat the ad exchange the same as the Supply-Side Platform, or SSP, even though it is a bit more complicated in reality. Thus, the main task of ad exchanges is running ad auctions. When SSP receives a bid request for an ad from a Publisher with information about a page, ad space and user, it does the following steps:
-
First, it forwards the ad request from Publisher to all the interested ad agencies and collects their replies, which usually contain information about the ad to be shown (a picture and a link where user should land after clicking on the ad) and the bid price that ad agency is willing to pay to show this ad to this user in this page at the moment. We will call these Bid Replies.
-
Then, SSP runs an auction — basically, selects the ad that bids the most, sends the winning ad to the publisher to be shown to the user, and charges the agency for the corresponding price.
If SSP is playing a first-price auction (the clearing price is the winner’s bid), then the result of the auction can be described in simple formulas:
winner ad = argmax bid_i
clearing price = max bid_i
Some share of the money SSP keeps for its services, another share goes to Publisher.
Demand-Side: DSP
Ad agencies that listen to ad exchanges’ auction announcements and reply with ads and bids are called Demand-Side Platforms, or DSPs. Sometimes, they are just called bidderssince their main task is to bid in the ad auction, preferably with reasonable bid amounts.
For the sake of simplicity, let’s consider that DSP works with only one advertiser that has only one ad, so it will be sending the same ad for participation in all SSP’s auctions. Then DSP only needs to compute the right bid for each particular auction.
Usually, DSP’s auction bid equals the amount of revenue DSP is expecting to get for showing the ad on average. For example, if an advertiser promises to pay DSP $20 for each purchase of a pair of shoes, DSP predicts the probability that this user, seeing this ad on this page, will buy a pair of shoes afterwards — and computes the auction bid as a product of $20×purchase probability.
Basically, DSP’s auction bid is a mathematical expectation of DSP’s revenue from showing the ad:
bid = E advertiser_payout = CPA * p(purchase | user, ad, page),
WhereEis a mathematical expectation, CPA is how much the advertiser will pay DSP for each desired action, and p is the probability of the user performing the action if they see this ad on this page at the moment.
However, DSP may work with thousands of advertisers and may have millions of ads that need to be ranked for each auction. There might be hundreds of thousands of auctions happening every second. The predicted probability of purchase should be computed quickly and efficiently but should be precise enough. Thus, to simplify even more, DSP is paying its own money for showing all the ads, and only gets paid by the advertiser when purchases happen — the cost of imprecise predictions becomes noticeable almost in real-time.
DSP usually charges some fraction of revenue flowing through it.
Demand-Side: The Advertiser
The Advertiser can also be called The Seller, as the goal of this business is to sell to users as many products as possible by spending reasonable money on advertising.
In our case, Advertiser is a shoe brand. The brand knows how to manufacture shoes and knows that each pair of shoes sold will bring it $20 dollars. So, Advertiser hires an ad agency and tells them to bring as many purchases as they can, as long as Advertiser is paying them less than $20 per purchase.
Advertiser is paying Publisher for ads being shown, with SSPs and DSPs getting their cut. Advertiser in turn gets paid by the user when the user makes a purchase in Advertiser’s shop.
Typical Examples and Other Business Models
This is an oversimplified view of the landscape, and in reality, there is a larger variety of roles. There are countless intermediaries at each step of the exchange. For example, ad networks aggregating and representing multiple publishers, or larger DSPs aggregating demand from smaller DSPs. There are companies routing requests between multiple SSPs and DSPs, sometimes enriching them with additional information, optimizing bid prices or detecting and removing anomaly traffic, like BidSwitch. There are also companies monetizing data about users called DMPs (Data Management Platforms).
Typical examples of advertisers can be Adidas, Mercedes Benz or Uber. Instances of DSP are Amazon Advertising Platform or Criteo. SSP representatives are PubMatic or OpenX. And typical publishers can be Washington Post, MSN and WordPress. One of the biggest DMPs is Oracle’s BlueKai.
Moreover, in reality, business roles frequently fuse together. For example, social networks like Twitter (newly X) and Facebook play the roles of publisher, SSP and DSP at the same time. Some SSPs act as DSPs by buying traffic from other SSPs, and some publishers act as DSPs by accepting advertising budgets from advertisers directly.
Businesses in programmatic advertising tend to grow and consolidate models over time. For example, Facebook also owns Facebook Ad Network, and Google, while initially being a publisher and an advertiser itself, over time acquired an SSP (Google Ads Manager, previously known as DoubleClick for Publishers or DFP, and AdX, previously known as DoubleClick Ad Exchange) and a DSP (DV360, previously known as DoubleClick Bid Manager, or DBM) products. Criteo, one of the biggest DSPs, in turn, recently announced an SSP called Commerce Grid.
Further Discussion
By design, due to the ad auction and ad personalization, RTB is the most efficient possible way of ad delivery since it shows ads that maximize the expected revenue flow in the ecosystem. The user always gets the best possible ads matching their interests, the ad gets shown to the most interested users — so that the amount of expected purchases is the highest (weighted by how much money advertisers are willing to pay for each purchase generated by ads). The expected amount of money to be paid by Advertisers is maximized, and therefore Publisher's revenue is maximized too.
winner ad = argmax bid_i = argmax E advertiser_payout_i
clearing price = max bid_i = max E advertiser_payout_i
RTB contrasts sharply with traditional advertising platforms like TV or billboards. While the latter methods might target broad demographics — placing billboards in specific neighborhoods or airing commercials on channels like the Discovery Channel to appeal to a certain interest group — they lack the capacity for nuanced personalization. Consequently, a significant portion of the audience is exposed to ads that hold little to no relevance for them.
Consider the scenario where a TV commercial for female cosmetics is viewed by both women and men. In this case, the advertiser effectively reaches only female viewers, missing the mark with the male audience. This mismatch can discourage advertisers catering to the male demographic from investing in that particular TV channel. Over time, this could result in reduced advertising expenditure on such platforms, leading to a dip in revenues for these TV channels.
However, the main existential challenge of programmatic advertising is that Publisher gets paid for every ad being shown, and the payments are usually based on expected purchases, but Advertiser only pays for rare purchase events that actually happened. Someone has to play the role of the buffer between predictions and reality, and in our oversimplified setup, this role is played by the DSP. It is one of the most important roles, since on the one hand, DSP is risking its own money by paying publishers for all ads and getting paid by the advertiser on conversions. It has to make sure that its predictions are correct, made within a strict time limit of hundreds of milliseconds, and that the cost of CPU is lower than DSP’s revenue from showing the ad, which makes the DSP’s task arguably the most challenging (we will discuss in the following articles how DSPs solve this challenge). On the other hand, improvements in the DSPs prediction engine are the most important growth driver for the whole ecosystem, as they lead to more effective ad allocation and generate more purchases and higher advertiser payouts.
To Be Continued…
In wrapping up this introductory article on programmatic advertising, we have taken a foundational look at the major players in this realm and touched upon various business models. As we delve deeper into subsequent articles, we'll closely examine each business, seeking to grasp the intricate challenges they face, both technically and mathematically. The realm of online advertising often pivots around measurable goals of revenue or cost. With feedback being tracked in real time, the implications of errors can be swiftly and accurately determined. This immediate and precise feedback, in turn, paves the way for the application of Machine Learning solutions.
