Journal of Artificial Intelligence Research · 2009 · 220 citations · 24 references
ComputingAutonomous SystemsComputer ScienceArtificial IntelligenceMachine LearningLearner AutonomyLearning From DemonstrationPolicy LearningReinforcement LearningReinforcement Learning FoundationsDemonstration LearningDemonstration-based Policy LearningDemonstration-based Reinforcement LearningInteractive Policy LearningInteractive Reinforcement LearningLearning From Demonstration With Safety ConstraintsReward Learning From DemonstrationTask Learning From Demonstration
We present Confidence-Based Autonomy (CBA), an interactive algorithm for policy learning from demonstration. The CBA algorithm consists of two components which take advantage of the complimentary abilities of humans and computer agents. The first component, Confident Execution, enables the agent to identify states in which demonstration is required, to request a demonstration from the human teacher and to learn a policy based on the acquired data. The algorithm selects demonstrations based on a measure of action selection confidence, and our results show that using Confident Execution the agent requires fewer demonstrations to learn the policy than when demonstrations are selected by a human teacher. The second algorithmic component, Corrective Demonstration, enables the teacher to correct any mistakes made by the agent through additional demonstrations in order to improve the policy and future task performance. CBA and its individual components are compared and evaluated in a complex simulated driving domain. The complete CBA algorithm results in the best overall learning performance, successfully reproducing the behavior of the teacher while balancing the tradeoff between number of demonstrations and number of incorrect actions during learning.
24
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
Selection of relevant features and examples in machine learning
Avrim Blum, Pat Langley · Artificial Intelligence · 1997 · 3.3K citations
A survey of robot learning from demonstration
Brenna Argall, Sonia Chernova, Manuela Veloso et al. · Robotics and Autonomous Systems · 2008 · 3.3K citations
Apprenticeship learning via inverse reinforcement learning
Pieter Abbeel, Andrew Y. Ng · 2004 · 2.9K citations