2020 · 22 citations · 14 references
ComputingComputer ScienceRobotics EngineeringEdge ComputingRobot ControlRoboticsEdge Artificial IntelligenceEdge DevicesFederated LearningMulti-agent Robot LearningPolicy LearningReinforcement LearningEdge Federated LearningFederated Learning For RobotsFederated Reinforcement LearningMulti-agent Policy LearningMulti-agent Reinforcement LearningPolicy Model TransferReinforcementReinforcement Learning For Robotics
Reinforcement learning has recently been studied in various fields and also used to optimally control real devices (e.g., robotic arms). In this paper, we try to allow multiple reinforcement learning agents to learn optimal control policy on their own devices of the same type but with slightly different dynamics. For such multiple devices, there is no guarantee that an agent who interacts only with one device and learns the optimal control policy will also control another device well. Therefore, we may need to apply independent reinforcement learning to each device individually, which requires time-consuming effort. To solve this problem, we propose a new federated reinforcement learning architecture where each agent working on its independent device shares their learning experience with each other, and transfers a mature policy model parameters into other agents. We incorporate the Actor-Critic PPO algorithm into each agent in the proposed collaborative architecture, and propose an efficient procedure for the gradient sharing and the model transfer. We also use edge computing to solve network problems that occur when training multiple real devices at the same time. Using multiple rotary inverted pendulum devices, we demonstrate that the proposed federated reinforcement learning scheme can effectively facilitate the learning process for multiple devices, and that the learning speed can be faster if more agents are involved.
14
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
Reinforcement Learning: An Introduction
Jeffrey D. Johnson, Jinghong Li, Zengshi Chen · Neurocomputing · 2000 · 8.7K citations