International Conference on Learning Representations · 2020 · 26 citations · 22 references
ComputingDecision SciencesArtificial IntelligenceMachine LearningMarkov Decision ProcessesExploration-exploitation TradeoffHeuristic SearchReinforcement LearningExploration In Reinforcement LearningLearning From Sparse RewardsSparse Reward Learning
While many option discovery methods have been proposed to accelerate exploration in reinforcement learning, they are often heuristic. Recently, covering options was proposed to discover a set of options that provably reduce the upper bound of the environment's cover time, a measure of the difficulty of exploration. Covering options are computed using the eigenvectors of the graph Laplacian, but they are constrained to tabular tasks and are not applicable to tasks with large or continuous state-spaces. We introduce deep covering options, an online method that extends covering options to large state spaces, automatically discovering task-agnostic options that encourage exploration. We evaluate our method in several challenging sparse-reward domains and we show that our approach identifies less explored regions of the state-space and successfully generates options to visit these regions, substantially improving both the exploration and the total accumulated reward.
22
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2016 · 6.8K citations · Full text
Computing, Computer Science, Physics +17
MuJoCo: A physics engine for model-based control
Emanuel Todorov, Tom Erez, Yuval Tassa · 2012 · 4.4K citations
Engineering, Mechanical Engineering, Robotics Engineering +17
Algebraic connectivity of graphs
Miroslav Fiedler · Czechoslovak Mathematical Journal · 1973 · 4K citations · Full text
Kybernetes · 1998 · 1.6K citations