Reinforcement Learning-Based Approximate Optimal Control for Attitude Reorientation Under State Constraints

Abstract

This article addresses the attitude reorientation problems of rigid bodies under multiple state constraints. A novel reinforcement learning (RL)-based approximate optimal control method is proposed to make the tradeoff between control cost and performance. The novelty lies in that it guarantees constraint handling abilities on attitude forbidden zones and angular velocity limits. To achieve this, barrier functions are employed to encode the constraint information into the cost function. Then, an RL-based learning strategy is developed to approximate the optimal cost function and control policy. A simplified critic-only neural network (NN) is employed to replace the conventional actor-critic structure once adequate data are collected online. This design guarantees the uniform boundedness of reorientation errors and NN weight estimation errors subject to the satisfaction of a finite excitation condition, which is a relaxation compared with the persistent excitation condition that is typically required for this class of problems. More importantly, all underlying state constraints are strictly obeyed during the online learning process. The effectiveness and advantages of the proposed controller are verified by both numerical simulations and experimental tests based on a comprehensive hardware-in-loop testbed.