Efficient bayesian hierarchical user modeling for recommendation system

TLDR

Personalized recommendation systems learn user profiles from feedback, but scaling to millions of users is challenging and EM algorithms converge slowly on sparse data. The paper proposes a fast learning technique to train many individual user profiles efficiently. The authors develop a fast learning algorithm that optimizes the joint likelihood over millions of users, reducing computational cost. The algorithm’s effectiveness and efficiency are validated theoretically and empirically on Netflix and MovieLens data.

Abstract

A content-based personalized recommendation system learns user specific profiles from user feedback so that it can deliver information tailored to each individual user's interest. A system serving millions of users can learn a better user profile for a new user, or a user with little feedback, by borrowing information from other users through the use of a Bayesian hierarchical model. Learning the model parameters to optimize the joint data likelihood from millions of users is very computationally expensive. The commonly used EM algorithm converges very slowly due to the sparseness of the data in IR applications. This paper proposes a new fast learning technique to learn a large number of individual user profiles. The efficacy and efficiency of the proposed algorithm are justified by theory and demonstrated on actual user data from Netflix and MovieLens.