MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction

TLDR

Predicting future behavior of road users is a critical challenge in autonomous driving, requiring deep learning models to fuse heterogeneous perception signals and map data while inferring highly multimodal future distributions. The paper introduces MultiPath++, a future prediction model that achieves state‑of‑the‑art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture by using sparse encoding of polylines and raw agent states, a context‑aware fusion with a reusable multi‑context gating component, learning latent anchor embeddings end‑to‑end, and exploring ensembling and output aggregation techniques for probabilistic multimodal outputs. Extensive ablation experiments demonstrate that MultiPath++ achieves state‑of‑the‑art performance on the Argoverse Motion Forecasting Competition and the Waymo Open Dataset Motion Prediction Challenge.

Abstract

Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing heterogeneous world state in the form of rich perception signals and map information, and inferring highly multi-modal distributions over possible futures. In this paper, we present MultiPath++, a future prediction model that achieves state-of-the-art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture [34] by revisiting many design choices. The first key design difference is a departure from dense image-based encoding of the input world state in favor of a sparse encoding of heterogeneous scene elements: MultiPath++ consumes compact and efficient polylines to describe road features, and raw agent state information directly (e.g., position, velocity, acceleration). We propose a context-aware fusion of these elements and develop a reusable multi-context gating fusion component. Second, we reconsider the choice of pre-defined static anchors, and develop a way to learn latent anchor embeddings end-to-end in the model. Lastly, we explore ensembling and output aggregation techniques—common in other ML domains—and find effective variants for our probabilistic multimodal output representation. We perform an extensive ablation on these design choices, and show that our proposed model achieves state-of-the-art performance on the Argoverse Motion Forecasting Competition [10] and the Waymo Open Dataset Motion Prediction Challenge [13].