MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction

by   Balakrishnan Varadarajan, et al.

Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing heterogeneous world state in the form of rich perception signals and map information, and inferring highly multi-modal distributions over possible futures. In this paper, we present MultiPath++, a future prediction model that achieves state-of-the-art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture by revisiting many design choices. The first key design difference is a departure from dense image-based encoding of the input world state in favor of a sparse encoding of heterogeneous scene elements: MultiPath++ consumes compact and efficient polylines to describe road features, and raw agent state information directly (e.g., position, velocity, acceleration). We propose a context-aware fusion of these elements and develop a reusable multi-context gating fusion component. Second, we reconsider the choice of pre-defined, static anchors, and develop a way to learn latent anchor embeddings end-to-end in the model. Lastly, we explore ensembling and output aggregation techniques – common in other ML domains – and find effective variants for our probabilistic multimodal output representation. We perform an extensive ablation on these design choices, and show that our proposed model achieves state-of-the-art performance on the Argoverse Motion Forecasting Competition and the Waymo Open Dataset Motion Prediction Challenge.


page 1

page 2

page 3

page 4


DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets

Due to the stochasticity of human behaviors, predicting the future traje...

DenseTNT: Waymo Open Dataset Motion Prediction Challenge 1st Place Solution

In autonomous driving, goal-based multi-trajectory prediction methods ar...

MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction

Predicting human behavior is a difficult and crucial task required for m...

Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Prediction

Forecasting the future behavior of all traffic agents in the vicinity is...

Wayformer: Motion Forecasting via Simple Efficient Attention Networks

Motion forecasting for autonomous driving is a challenging task because ...

ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposals

Motion forecasting is a key module in an autonomous driving system. Due ...

PrognoseNet: A Generative Probabilistic Framework for Multimodal Position Prediction given Context Information

The ability to predict multiple possible future positions of the ego-veh...

Please sign up or login with your details

Forgot password? Click here to reset