arXiv (Cornell University) · 2017 · 22 citations · 32 references
Artificial IntelligenceAgent-based SystemCognitive ScienceEngineeringAgent Decision-makingAutomated ReasoningAutonomous LearningProtocol ProgramsHuman-in-the-loop Machine LearningAction Model LearningAutonomous Agent SystemComputer ScienceIntelligent SystemsRobot LearningMulti-agent LearningReward ShapingAgent-agnostic Schema
Providing RL agents with expert advice can dramatically improve learning, yet prior teaching protocols are tailored to specific agent representations, offering effective but specialized procedures. This work introduces protocol programs, an agent‑agnostic schema for Human‑in‑the‑Loop RL, aiming to embed human teacher benefits without assuming agent internals. Protocol programs represent existing methods such as action pruning, reward shaping, and simulation training as special cases, and the authors conduct preliminary experiments on simple domains. Preliminary experiments on simple domains show that the schema can subsume action pruning, reward shaping, and simulation training.
Providing Reinforcement Learning agents with expert advice can dramatically improve various aspects of learning. Prior work has developed teaching protocols that enable agents to learn efficiently in complex environments; many of these methods tailor the teacher's guidance to agents with a particular representation or underlying learning scheme, offering effective but specialized teaching procedures. In this work, we explore protocol programs, an agent-agnostic schema for Human-in-the-Loop Reinforcement Learning. Our goal is to incorporate the beneficial properties of a human teacher into Reinforcement Learning without making strong assumptions about the inner workings of the agent. We show how to represent existing approaches such as action pruning, reward shaping, and training in simulation as special cases of our schema and conduct preliminary experiments on simple domains.
32