Memory-based crowd-aware robot navigation using deep reinforcement learning

Abstract

Abstract The evolution of learning techniques has led robotics to have a considerable influence in industrial and household applications. With the progress in technology revolution, the demand for service robots is rapidly growing and extends to many applications. However, efficient navigation of service robots in crowded environments, with unpredictable human behaviors, is still challenging. The robot is supposed to recognize surrounding information while navigating, and then act accordingly. To address this issue, the proposed method crowd Aware Memory-based Reinforcement Learning (CAM-RL) uses gated recurrent units to store the relative dependencies among the crowd, and utilizes the human–robot interactions in the reinforcement learning framework for collision-free navigation. The proposed method is compared with the state-of-the-art techniques of multi-agent navigation, such as Collision Avoidance with Deep Reinforcement Learning (CADRL), Long Short-Term Memory Reinforcement Learning (LSTM-RL) and Social Attention Reinforcement Learning (SARL). Experimental results show that the proposed method can identify and learn human–robot interactions more extensively and efficiently than above-mentioned methods while navigating in a crowded environment. The proposed method achieved a success rate of greater than or equal to $$99\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>99</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> and a collision rate of less than or equal to $$1\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>1</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> in all test case scenarios, which is better compared to the previously proposed methods.