Automatic ad format selection via contextual bandits

TLDR

Visual design of online display ads influences click‑through rate and revenue, and selecting a layout requires balancing exploration of new designs against exploitation of known effective ones, yet testing every bandit policy on live traffic is impractical. The study frames automatic ad layout selection as a contextual bandit problem to balance exploration and exploitation. We developed an offline replayer to evaluate bandit algorithms on historical data and benchmark several common bandit policies. Offline replay, or exploration scavenging, accurately estimates ad layout policy performance at LinkedIn using only historical data.

Abstract

Visual design plays an important role in online display advertising: changing the layout of an online ad can increase or decrease its effectiveness, measured in terms of click-through rate (CTR) or total revenue. The decision of which lay- out to use for an ad involves a trade-off: using a layout provides feedback about its effectiveness (exploration), but collecting that feedback requires sacrificing the immediate reward of using a layout we already know is effective (exploitation). To balance exploration with exploitation, we pose automatic layout selection as a contextual bandit problem. There are many bandit algorithms, each generating a policy which must be evaluated. It is impractical to test each policy on live traffic. However, we have found that offline replay (a.k.a. exploration scavenging) can be adapted to provide an accurate estimator for the performance of ad layout policies at Linkedin, using only historical data about the effectiveness of layouts. We describe the development of our offline replayer, and benchmark a number of common bandit algorithms.