SEARCH ADVERTISING

If, as Goldfarb (2014) says, dramatically improved targeting capabilities are the most important feature distinguishing Internet advertising from advertising in other media, then search engines’ pairings of ads with consumers who have self-identified as interested in related products or services should represent the pinnacle of what can be accomplished through targeting with Internet ads.

Reflecting Hal Varian’s observation in Chapter 18 in this volume that search engines ‘offer a fertile ground for economic analysis’, a very large literature on search engine economics has developed during the relatively short time that search engines have existed in something approximating their current form. Because the questions that must be answered to improve our understanding of search engines are many, this literature is also quite diverse. In this section, we focus on key findings and unresolved questions for the most prominent lines of research.

20.4.1 Background

A search engine faces the challenge of simultaneously setting prices for each of the multi­ple paid ad positions on a search page. As a general rule, paid ads positioned immediately above the unpaid listings on a search page are expected to generate more clicks for a search ad than positions to the right of the unpaid listings and the contribution to an ad’s click count increases with the height of its position on the page.

There is still debate over the reasons GSP auctions have become the de facto standard for setting prices for search ad positions. One of the ostensible reasons offered for the switch from first-price auctions to GSP auctions was that equilibria for GSP auctions are more stable. However, recent empirical studies (Zhang and Feng, 2011; Yuan, 2012) do not support this claim. Yuan (2012) used data from before and after Yahoo! switched from a generalized first-price (GFP) auction to an unweighted GSP auction and found that after the switch advertisers changed their bids more often and the magnitudes of price changes were greater. Zhang and Feng (2011) observed sawtooth-like price cycles for both GFP and GSP auctions. On the other hand, Yuan’s (2011) study of comparative efficiency for the two auction mechanisms found that efficiency (measured by the fre­quency with which higher value advertisers were assigned better positions) was 4 percent higher for the GSP.

The literature on search engine economics that has developed since the introduction of the GSP auction has focused largely on GSP auction properties and the implications of its use for market outcomes, including market efficiency and the relative payoffs to the various participants in search markets. GSP auctions have a number of features that search engines can adjust to change the character of the auctions they run and these choices have implications for market performance. Bid ranking formulas and reserve prices are two that have received considerable attention in the literature.

20.4.1.1 Bid ranking

An advertiser’s bid is the amount it offers to pay for each click generated by its ad and when search engines first started using GSP auctions advertisers were ranked by their bids alone. A GSP auction that ranks bids in this manner is referred to as an unweighted auction. But the major search engines fairly quickly switched to ‘rank-by-revenue’ formulas (also called click-weighted formulas) that assign ads positions based on what can roughly be described as (search engines’) predictions of their revenue-generating potential calculated as price per click times predicted number of clicks. Because an ad’s content and its position on a search page both influence the number of clicks it receives, it is common to think of the number of clicks received by an ad as the product of a posi­tion effect and an ad-specific effect that is commonly referred to as an ad’s relevance. The ‘rank-by-revenue’ rule employs bid weights proportional to predicted relevance and assigns each bid a rank based on the product of its bid and predicted relevance.

Of course search engines are free to employ bid weights that differ from those used for unweighted auctions and rank-by-revenue auctions and the effects of different bid weights on market performance and search engine revenue have been a matter of consid­erable interest. A family of ranking rules introduced by Lahaie and Pennock (2007) has probably received the most attention by researchers examining alternative ranking rules, possibly because its functional form is easily incorporated in economic models of search markets. For b_i the bid submitted by advertiser i, q_i the relevance of i’s ad and ‘squash­ing factor’ s, i’s bid is given a score of b_iq_i^s. All other bids are similarly scored and ranks determined accordingly. In theory there are no bounds on the values s might take, but it is often assumed that s ∈ [0, 1]. Squashing factors of 0 and 1 correspond, respectively, to unweighted bids and rank-by-revenue bids.

20.4.1.2 Reserve prices

Although they did not at the beginning, GSP auctions now typically include as a restric­tion a minimum price that must be paid by an advertiser to secure a position on a search page. Reserve prices can be either weighted or unweighted. An unweighted reserve price is simply a minimum nominal price per click that the advertiser occupying the lowest paid ad position on a search page must pay. If the reserve price is weighted, advertiser-specific relevance weights are applied to a nominal reserve price. Reserve prices can increase a search engine’s revenue by increasing the bid required to secure the lowest position, which in turn can raise the bids and prices paid for higher positions. In fact, Lucier et al. (2012) show that for a Bayesian model of GSP auctions a positive reserve price may be required to eliminate zero revenue equilibria and Thompson and Leyton-Brown (2013) show that reserve prices can be combined with a squashing factor of less than one to further increase search engine revenue. They also find that unweighted reserve prices outperform weighted reserve prices if enhancing search engine revenue is the objective.

20.4.2 Search Market Equilibria

The economics literature on search advertising markets has focused primarily on the nature of search market equilibria, with efficiency, comparative payoffs to search engines and advertisers and market stability the primary topics of investigation. The majority of the work to date has focused on formal models of search market equilibria, although the analyses based on these models are often helpfully informed by data and empirical studies of search markets.

GSP auctions present a number of challenges to model builders. For one thing, GSP auctions do not compel truthful bids (bids that reveal bidders true valuations for ad posi­tions). Given a range of bids and a limited number of bidders, within a finite range a rep­resentative bidder will be able to change its bid without affecting the position it secures or its payment.

The fact an advertiser’s bid determines the price paid by the advertiser occupying the next higher position also means that an advertiser so inclined could set its own bid near the top of the range that leaves its own position and price unchanged just to increase a rival bidder’s costs, a strategy known as malicious bidding. Liang and Qi (2007) show that search markets may not always converge when bidding is malicious. As a general matter, the fact that advertisers’ best response bids can be selected from a range of values leaves open the possibility that bidding strategies selected will not support convergence to stable equilibria (Cary et al., 2007; Zhou and Lukose, 2007).

20.4.2.1 Full information models

Similar simultaneous-move, full-information models developed by Edelman et al. (2007) and Varian (2007) constituted a breakthrough in the modeling of search market equi­libria and have served as touchstones for subsequent analysis. The breakthrough was a refinement to the Nash auction equilibrium that restricts the equilibria considered to those for which, given the prices associated with the positions, each advertiser prefers the position he or she has acquired to all other positions. The resulting set of equilibria, called ‘envy free’ by Edelman et al. (2007) and ‘symmetric Nash’ by Varian (2007), is still an infinite set, but all members of this set have attractive stability properties and the envy-free restriction substantially simplifies the derivation of equilibrium properties.

Lahaie and Pennock (2007) elaborate on the Varian (2007) and Edelman et al. (2007) models by adding the squashing factors described above to the bid ranking formula and show that when relevance and bids are positively correlated, a search engine can increase its revenue relative to what would be generated by a pure rank-by-revenue rule by selecting a squashing factor of less than 1. However, market efficiency is reduced when this strategy is employed. Athey and Nekipelov (2011) show that this finding holds for a model with less than fully informed bidders.

20.4.2.2 Incomplete information models

The fully informed bidders assumption has been criticized as unrealistic. Even for an unweighted GSP auction, a bidder would acquire direct knowledge of only the per click price paid by the bidder claiming the position immediately above his or hers, because it is his or her bid, and the bid of the advertiser occupying the position imme­diately below his or hers, which is the per click price he or she pays. The ad-specific weights applied to bids in weighted GSP auctions are not revealed by search engines. The standard argument justifying use of full information bidding models is that advertisers engage in repeated rounds of bidding against the same competitors and this gives them opportunities to vary their bids and observe the resulting allocation of positions. With the information gained through this process they can form relatively accurate estimates of their competitors’ bids and bid weights. Full information critics respond that a number of factors make this information difficult to acquire in real search markets where advertisers adjust their bids less frequently than auctions occur, advertisers with different ad budgets participate in auctions with different frequencies, and the mix of competing bidders changes from auction to auction. Edelman and Ostrovsky (2007) and Athey and Nekipelov (2011) provide evidence that search ads’ positions vary across auctions and Edelman and Ostrovsky (2007) report that bids fluctuate as well.

For agents with less than full information, Varian (2007) showed how GSP auctions could be modeled as Bayes-Nash equilibria and Thompson and Leyton-Brown (2013) employed such a model to examine the effects of squashing and reserve prices on search engine revenues, as described above. Gomes and Sweeney (2014) have probably provided the most thorough exploration of the properties of Bayes-Nash search equilibria. The critical simplifying assumptions underlying their model are that advertisers’ valuations for clicks are known only to themselves, these valuations are drawn from independent but identical distributions that are common knowledge to all advertisers, the value of a click to an advertiser is independent of the position of the advertiser’s ad, and the likeli­hood that exposure to an ad will elicit a click is the same for all advertisers. It is worth noting that the assumption regarding the distributions from which bids are drawn might be susceptible to some of the same criticisms levied against the full information models and the assumption that exposure to its ad has the same probability of being followed by a click for all advertisers rules out advertiser-specific relevance weights.

Gomes and Sweeney’s primary result identifies a necessary and sufficient condition for an efficient Bayes-Nash equilibrium to exist for a multi-position auction. Expressed in terms of positions’ click-through rates, the click-through rate associated with each position must exceed the click-through rate of the next lower position by at least the non-negligible amount required to ensure an advertiser with a higher valuation does not find it profitable to shift to the next lower position by undercutting the bid of the lower- valuation advertiser occupying that spot. Because nothing guarantees that this condition will be satisfied in search markets, there is no guarantee that an efficient Bayes-Nash equilibrium will exist. Gomes and Sweeney (2014) point out that their model is a static model of a single-shot game and suggest that further advances may be made by develop­ing dynamic models of search market equilibria.

20.4.2.3 A closer look at consumers’ influence on equilibria

The research discussed so far has focused primarily on interactions between advertis­ers and search engines. In a notable departure, Athey and Ellison (2011) assume that search users incur a positive cost each time they click on an ad and show that positive search costs can change the character of a search equilibrium considerably. Modeling search auctions as common value auctions, the authors find that in equilibrium advertis­ers are allocated positions according to their relevance, that this allows users to infer ads’ relevance rankings from their positions, and that advertisers submit bids that are mono­tone in relevance. Athey and Ellison also find that a positive reserve price can increase search engine profits and social welfare by keeping low-relevance ads off search pages and that an unweighted reserve price works better than a weighted reserve price in this regard.

20.4.3 There is Still Much to Do

From the literature reviewed in this section, it should be clear that we have come a long way in developing a better understanding of how search advertising markets operate, but that we also have a long way to go before we can claim with confidence that the models of search markets we work with provide reasonably accurate depictions of the way search markets really work, how efficiently they operate, or whether they are inherently stable. Work to develop upper-bound estimates on the extent to which the efficiency of search market equilibria might depart from the efficient market outcome is helpful when assess­ing the efficiency with which search markets operate (see, e.g., Caragiannis et al., 2012; Paes Leme and Tardos, 2010), but it is important to recognize that these estimates vary with the theoretical model that is employed to describe how search advertising markets operate. Better empirical grounding is needed to help with the assessment of how closely the assumptions of various search market models approximate conditions in real search markets.

Athey and Ellison’s (2011) model of search advertising markets in which clicks are costly to consumers highlights the importance of looking more closely at how consum­ers use and experience search engines and more work of this nature is needed. It is also important that we do not lose sight of the fact that the suppliers of the unpaid listings in the center of search pages are not passive participants in search advertising markets. Many of these listings are also ads and it is not unusual for an advertiser to have a paid ad and an unpaid listing on the same search page. Suppliers of unpaid listings work actively to improve their positions on search pages and strategies for doing so are actively discussed in trade press articles on ‘search engine optimization’ (SEO). The Huffington Post’s success in rapidly establishing a position as a major online media player has been attributed by some to its SEO skills (Shafer, 2007). There is also a tiny academic literature on SEO strategy (see, e.g., Parikh and Deshmukh, 2013). How can we know how effi­ciently search advertising markets operate if we don’t know whether advertisers consider paid and unpaid listings to be substitutes or complements or whether efforts by suppliers of unpaid listings to improve their positions on search pages serve consumers’ interests or not?

20.5