Item popularity and recommendation accuracy

TLDR

Recommendations from the long tail of item popularity are valuable, yet accuracy tends to decline toward the tail. This paper quantitatively examines the trade‑off between item popularity and recommendation accuracy and motivates a refinement of collaborative‑filtering training. Assuming a selection bias toward popular items, the authors define a tunable accuracy measure that can be gradually shifted toward the long tail. The new measure yields nearly unbiased accuracy estimates, and experiments show users appreciate only a small bias toward less popular items.

Abstract

Recommendations from the long tail of the popularity distribution of items are generally considered to be particularly valuable. On the other hand, recommendation accuracy tends to decrease towards the long tail. In this paper, we quantitatively examine this trade-off between item popularity and recommendation accuracy. To this end, we assume that there is a selection bias towards popular items in the available data. This allows us to define a new accuracy measure that can be gradually tuned towards the long tail. We show that, under this assumption, this measure has the desirable property of providing nearly unbiased estimates concerning recommendation accuracy. In turn, this also motivates a refinement for training collaborative-filtering approaches. In various experiments with real-world data, including a user study, empirical evidence suggests that only a small, if any, bias of the recommendations towards less popular items is appreciated by users.