Proceedings of the AAAI Conference on Artificial Intelligence · 2021 · 23 citations · 43 references
BusinessDecision SciencesMachine LearningMarkov Decision ProcessesRecommender SystemsExploration-exploitation TradeoffReinforcement LearningValue FunctionsModel-based Reinforcement LearningModel-free Reinforcement LearningReinforcement Learning ModelsReinforcement Learning TheoryStochastic DynamicsValue Function EstimationValue Function Generalization
In recent years, there are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems (RS). In this paper, we summarize three key practical challenges of large-scale RL-based recommender systems: massive state and action spaces, high-variance environment, and the unspecific reward setting in recommendation. All these problems remain largely unexplored in the existing literature and make the application of RL challenging. We develop a model-based reinforcement learning framework, called GoalRec. Inspired by the ideas of world model (model-based), value function estimation (model-free), and goal-based RL, a novel disentangled universal value function designed for item recommendation is proposed. It can generalize to various goals that the recommender may have, and disentangle the stochastic environmental dynamics and high-variance reward signals accordingly. As a part of the value function, free from the sparse and high-variance reward signals, a high-capacity reward-independent world model is trained to simulate complex environmental dynamics under a certain goal. Based on the predicted environmental dynamics, the disentangled universal value function is related to the user's future trajectory instead of a monolithic state and a scalar reward. We demonstrate the superiority of GoalRec over previous approaches in terms of the above three practical challenges in a series of simulations and a real application.
43
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2016 · 6.8K citations · Full text
Computing, Computer Science, Physics +17
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2015 · 5.4K citations · Full text
Computing, Computer Science, Physics +17
Playing Atari with Deep Reinforcement Learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · arXiv (Cornell University) · 2013 · 5.1K citations · Full text
Wide & Deep Learning for Recommender Systems
Heng-Tze Cheng, Levent Koç, Jeremiah Harmsen et al. · 2016 · 3.3K citations · Full text