Evaluating recommender systems

Abstract

Recommender systems now tend to gain popularity and significance. The proliferation of many recommender systems leads to the difficulty of locating a good recommender system. The algorithms contained in the recommender system determine the efficiency of the recommender systems. The question now is to find the most appropriate algorithms to meet users' needs. So far, the research carried out has focused on improving the accuracy of recommender systems. In this paper, we propose that the recommender system should move beyond the conventional accuracy criteria and take some other criteria into account, such as coverage, diversity, serendipity, scalability, adaptability, risk, novelty and so on. Experimental results with data from VELO indicate that people with different interest degree tend to prefer different algorithms; thus the use of various evaluation criteria to judge the performance of algorithm is meaningful.