Neural Information Processing Systems · 2019 · 61 citations · 15 references
ComputingComputer ScienceArtificial IntelligenceMachine LearningMarkov Decision ProcessesAdaptive LearningOnline AlgorithmsPolicy LearningReinforcement LearningTransfer LearningAuxiliary Task LearningEfficiencyOnline Machine LearningReinforcement Learning TransferSample-efficient Reinforcement LearningTask Weighting
Reinforcement learning is known to be sample inefficient, preventing its application to many real-world problems, especially with high dimensional observations like images. Transferring knowledge from other auxiliary tasks is a powerful tool for improving the learning efficiency. However, the usage of auxiliary tasks has been limited so far due to the difficulty in selecting and combining different auxiliary tasks. In this work, we propose a principled online learning algorithm that dynamically combines different auxiliary tasks to speed up training for reinforcement learning. Our method is based on the idea that auxiliary tasks should provide gradient directions that, in the long term, help to decrease the loss of the main task. We show in various environments that our algorithm can effectively combine a variety of different auxiliary tasks and achieves significant speedup compared to previous heuristic approches of adapting auxiliary task weights.
15
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Mastering the game of Go without human knowledge
David Silver, Julian Schrittwieser, Karen Simonyan et al. · Nature · 2017 · 9.2K citations
Engineering, Educational Psychology, Cognitive Psychology +7
Rich Caruana · Machine Learning · 1997 · 6.3K citations · Full text