MySpace Video Recommendation with Map-Reduce on Qizmt

Abstract

Recent years have seen a surge in online video content which is often used as a communication medium and information resource by users. The explosive growth in content has given rise to the need of developing effective recommendation system which can help users discover meaningful and interesting videos. In this paper, we present a large-scale Map-Reduce video recommendation system. Our approach includes item-to-item collaborative filtering using video views data, and involves content analysis of video metadata to extract feature representation for identifying similar videos for recommendation. Recommendation results are further filtered through a refinement stage using semantic similarity. As an integrated pipeline, we show how our proposed approach is implemented in Qizmt which is a. Net MapReduce framework. Additionally, our approach is capable of updating video recommendation index with hourly added video data. We describe our recommendation approach using a portion (23 million) of all videos from My Space and undertake quantitative as well as qualitative evaluation.