Measurement and Control · 2021 · 19 citations · 11 references
ComputingComputer ScienceRobotics EngineeringReal-time SystemsRobot ControlRoboticsControl SystemsDeep LearningReal-time ControlReinforcement LearningSimulation-based LearningControl System ModelingDeep Reinforcement LearningExperience ReplayNonlinear Control SystemsNonlinear SystemsReinforcementReplayTraining
For real applications, rotary inverted pendulum systems have been known as the basic model in nonlinear control systems. If researchers have no deep understanding of control, it is difficult to control a rotary inverted pendulum platform using classic control engineering models, as shown in section 2.1. Therefore, without classic control theory, this paper controls the platform by training and testing reinforcement learning algorithm. Many recent achievements in reinforcement learning (RL) have become possible, but there is a lack of research to quickly test high-frequency RL algorithms using real hardware environment. In this paper, we propose a real-time Hardware-in-the-loop (HIL) control system to train and test the deep reinforcement learning algorithm from simulation to real hardware implementation. The Double Deep Q-Network (DDQN) with prioritized experience replay reinforcement learning algorithm, without a deep understanding of classical control engineering, is used to implement the agent. For the real experiment, to swing up the rotary inverted pendulum and make the pendulum smoothly move, we define 21 actions to swing up and balance the pendulum. Comparing Deep Q-Network (DQN), the DDQN with prioritized experience replay algorithm removes the overestimate of Q value and decreases the training time. Finally, this paper shows the experiment results with comparisons of classic control theory and different reinforcement learning algorithms.
11
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Deep Reinforcement Learning with Double Q-Learning
Hado van Hasselt, Arthur Guez, David Silver · 2016 · 6K citations · Full text
Proceedings of Thirtieth AAAI Conference on Artificial Intelligence
Lukáš Chrpa, Bart De Schutter, Daniele Magazzeni et al. · 2016 · 674 citations