Sussex Research Online (University of Sussex) · 2020 · 64 citations · 56 references
Open access
ComputingComputer ScienceNeuroscienceArtificial IntelligenceComputational NeuroscienceMachine LearningAction RepresentationBayesian InferenceDecision-making ModelsExploration-exploitation TradeoffReinforcement LearningReinforcement Learning FoundationsActive InferenceBayesian EvidenceComputational Neuroscience ModelsEfficiencyExplorationReinforcement Learning ModelsSample EfficiencySample-efficient Reinforcement Learning
In reinforcement learning (RL), agents often operate in partially observed and uncertain environments. Model-based RL suggests that this is best achieved by learning and exploiting a probabilistic model of the world. ‘Active inference’ is an emerging normative framework in cognitive and computational neuroscience that offers a unifying account of how biological agents achieve this. On this framework, inference, learning and action emerge from a single imperative to maximize the Bayesian evidence for a niched model of the world. However, implementations of this process have thus far been restricted to low-dimensional and idealized situations. Here, we present a working implementation of active inference that applies to high-dimensional tasks, with proof-of-principle results demonstrating efficient exploration and an order of magnitude increase in sample efficiency over strong model-free baselines. Our results demonstrate the feasibility of applying active inference at scale and highlight the operational homologies between active inference and current model-based approaches to RL.
56
Auto-Encoding Variational Bayes
Diederik P. Kingma, Max Welling · UvA-DARE (University of Amsterdam) · 2013 · 15.6K citations · Full text
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2016 · 6.8K citations · Full text
Computing, Computer Science, Physics +17