Towards a more realistic evaluation

Abstract

The use of temporal dynamic terms in Matrix Factorization (MF) models of recommendation have been proposed as a means to obtain better accuracy in rating prediction task. However, the way such models have been tested may not be a realistic setting for recommendation. In this paper, we evaluated rating prediction and top-N recommendation tasks using a MF model with and without temporal dynamic terms under two evaluation settings. Our experiments show that the addition of dynamic parameters do not necessarily yield to better results on these tasks when a more strict time-aware separation of train/test data is performed, and moreover, results may vary notably when different evaluation schemes are used.