Predicting Skill Learning in a Large, Longitudinal MOBA Dataset

Abstract

The exploration of the relationships between behavior and cognitive psychology of game players has gained impetus in recent years because such links provides an opportunity for improving user experiences and optimizing products in the games industry. At the same time, the volume and global scope of digital game telemetry data has opened up new experimental opportunities for studying human behavior at large scales. Prior research has demonstrated that a relation exists between learning rates and performance. Although many factors might contribute to this correlation at least one may be the presence of innate cognitive resources, as demonstrated in recent work relating IQ and performance in a Multi-player Online Battle Arena game. Here, we extend this work by examining the relationship between early learning rate and long term performance using a 400,000 player longitudinal dataset generated by new players of the widely-played MOBA League of Legends. We observed that the learning rate of new players in a competitive season explains a significant amount of variance in the performance at the end of the year. This analysis was then extended by training two multivariate classifiers (Logistic Regression, Random Forest) for predicting players who by the end of the season would be considered masters (top 0.05%), based on their performance in the first 10 matches of the same season. Both classifiers performed similarly (ROC AUC 0.888 for Logistic Regression, 0.878 for Random Forest), extending the time frame for skill prediction in games based on a relatively sparse sample of early data. We discuss the implications for these findings based on preexisting psychological studies of learning and intelligence, and close with challenges and direction for future research.