<i>MHRR</i>: MOOCs Recommender Service With Meta Hierarchical Reinforced Ranking

Abstract

The exponential growth of Massive Open Online Courses (MOOCs) surges the needs of advanced models for personalized Online Education Services (OES). Existing solutions successfully recommend MOOCs courses via deep learning models, they however generate weak “course embeddings” with original profiles, which contain noisy and few enrolled courses. On the other hand, existing algorithms provide recommendation orders according to the score of each course while ignoring personalized demands of users. To tackle the above challenges, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>M</u>eta <u>H</u>ierarchical <u>R</u>einforced <u>R</u>anking</i> approach <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MHRR</i> , which consists of a meta hierarchical reinforcement learning pre-trained mechanism and an over-parameterized ranking regressor to enhance the representation learning of courses and learners while refining the ranking result of recommended courses. Specifically, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MHRR</i> combines a user profile reviser and a meta embedding generator to provide course embedding representation enhancement for recommender services. Furthermore, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MHRR</i> transforms learned representations generated from recommender services with Gaussian kernel approximation to over-parameterize the downstream learning to rank (LTR) models with representations in ultra-high dimensionality. We deploy <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MHRR</i> on a real-world MOOCs platform and evaluate it with a large number of baseline models. The results show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MHRR</i> outperforms baseline algorithms on two major metrics, including Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG). Also, we conduct a 7-day online evaluation using the realistic traffic of a large-scale real-world MOOCs platform, where we can still observe significant improvement in real-world applications. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MHRR</i> performs consistently both in the online and offline evaluation.