Leveraging Missing Ratings to Improve Online Recommendation Systems

TLDR

Product recommendation systems rely on sparse user ratings, yet most users rate only a few items and missing data are often treated as random, ignoring potential informative patterns. The authors propose a framework that jointly models the selection process and rating values to better exploit missing data in recommendation systems. On the EachMovie dataset, modeling selection and ratings together reduces holdout error by over 10 % and shows that missingness is strongly nonignorable, thereby improving recommendation quality.

Abstract

“Product recommendation systems” are backbones of the Internet economy, leveraging customers' prior product ratings to generate subsequent suggestions. A key feature of recommendation data is that few customers rate more than a handful of items. Extant models take missing recommendation rating data to be missing completely at random, implicitly presuming that they lack meaningful patterns or utility in improving ratings accuracy. For the widely studied EachMovie data, the authors find that missing data are strongly nonignorable. Recommendation quality is improved substantially by jointly modeling “selection” and “ratings,” both whether and how an item is rated. Accounting for missing ratings and various sources of heterogeneity offers a rich portrait of which items are rated well, which are rated at all, and how these processes are intertwined, while reducing holdout error by 10% or more. The authors discuss ways to implement the proposed framework within existing recommendation systems and its implications for marketers.