Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learning

TLDR

Autonomous vehicles must predict surrounding vehicles’ behavior probabilistically and interactively to drive safely and efficiently. The study proposes a hierarchical inverse reinforcement learning–based probabilistic prediction method for interaction‑aware driving behavior. The method models human driving as a hierarchical process of discrete and continuous decisions, learns a mixture of trajectory distributions via hierarchical IRL from real demonstrations, and is illustrated on a ramp‑merging scenario. Experiments demonstrate accurate prediction of both discrete decisions (yield or pass) and continuous trajectories.

Abstract

Autonomous vehicles (AVs) are on the road. To safely and efficiently interact with other road participants, AVs have to accurately predict the behavior of surrounding vehicles and plan accordingly. Such prediction should be probabilistic, to address the uncertainties in human behavior. Such prediction should also be interactive, since the distribution over all possible trajectories of the predicted vehicle depends not only on historical information, but also on future plans of other vehicles that interact with it. To achieve such interaction-aware predictions, we propose a probabilistic prediction approach based on hierarchical inverse reinforcement learning (IRL). First, we explicitly consider the hierarchical trajectory-generation process of human drivers involving both discrete and continuous driving decisions. Based on this, the distribution over all future trajectories of the predicted vehicle is formulated as a mixture of distributions partitioned by the discrete decisions. Then we apply IRL hierarchically to learn the distributions from real human demonstrations. A case study for the ramp-merging driving scenario is provided. The quantitative results show that the proposed approach can accurately predict both the discrete driving decisions such as yield or pass as well as the continuous trajectories.