IEEE Access · 2019 · 31 citations · 32 references
BusinessIndustrial EngineeringMarkov Decision ProcessesOptimizationSafety EngineeringConstrained Markov Decision ProcessesExploration-exploitation TradeoffOptimal ControlPolicy LearningReinforcement LearningStochastic OptimizationConstrained Reinforcement LearningExplorationMarkov Decision ModelsObjective Function FormulationSafe ExplorationSafe LearningSafe Reinforcement LearningSafety Constraints
The application of reinforcement learning in industrial fields makes the safety problem of the agent a research hotspot. Traditional methods mainly alter the objective function and the exploration process of the agent to address the safety problem. Those methods, however, can hardly prevent the agent from falling into dangerous states because most of the methods ignore the damage caused by unsafe states. As a result, most solutions are not satisfactory. In order to solve the aforementioned problem, we come forward with a safe Q-learning method that is based on constrained Markov decision processes, adding safety constraints as prerequisites to the model, which improves standard Q-learning algorithm so that the proposed algorithm seeks for the optimal solution ensuring that the safety premise is satisfied. During the process of finding the solution in form of the optimal state-action value, the feasible space of the agent is limited to the safe space that guarantees the safety via the feasible space being filtered by constraints added to the action space. Because the traditional solution methods are not applicable to the safe Q-learning model as they tend to obtain local optimal solution, we take advantage of the Lagrange multiplier method to solve the optimal action that can be performed in the current state based on the premise of linearizing constraint functions, which not only improves the efficiency and accuracy of the algorithm, but also guarantees to obtain the global optimal solution. The experiments verify the effectiveness of the algorithm.
32
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 9K citations · Full text
Tom Schaul, John Quan, Ioannis Antonoglou et al. · arXiv (Cornell University) · 2015 · 2K citations · Full text