A Hierarchical Hybrid Learning Framework for Multi-agent Trajectory Prediction

by   Yujun Jiao, et al.

Accurate and robust trajectory prediction of neighboring agents is critical for autonomous vehicles traversing in complex scenes. Most methods proposed in recent years are deep learning-based due to their strength in encoding complex interactions. However, unplausible predictions are often generated since they rely heavily on past observations and cannot effectively capture the transient and contingency interactions from sparse samples. In this paper, we propose a hierarchical hybrid framework of deep learning (DL) and reinforcement learning (RL) for multi-agent trajectory prediction, to cope with the challenge of predicting motions shaped by multi-scale interactions. In the DL stage, the traffic scene is divided into multiple intermediate-scale heterogenous graphs based on which Transformer-style GNNs are adopted to encode heterogenous interactions at intermediate and global levels. In the RL stage, we divide the traffic scene into local sub-scenes utilizing the key future points predicted in the DL stage. To emulate the motion planning procedure so as to produce trajectory predictions, a Transformer-based Proximal Policy Optimization (PPO) incorporated with a vehicle kinematics model is devised to plan motions under the dominant influence of microscopic interactions. A multi-objective reward is designed to balance between agent-centric accuracy and scene-wise compatibility. Experimental results show that our proposal matches the state-of-the-arts on the Argoverse forecasting benchmark. It's also revealed by the visualized results that the hierarchical learning framework captures the multi-scale interactions and improves the feasibility and compliance of the predicted trajectories.


LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and Trajectory Prediction

Multi-agent trajectory prediction is a fundamental problem in autonomous...

TSGN: Temporal Scene Graph Neural Networks with Projected Vectorized Representation for Multi-Agent Motion Prediction

Predicting future motions of nearby agents is essential for an autonomou...

ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptation

Forecasting future trajectories of agents in complex traffic scenes requ...

Multiple Futures Prediction

Temporal prediction is critical for making intelligent and robust decisi...

R-Pred: Two-Stage Motion Prediction Via Tube-Query Attention-Based Trajectory Refinement

Predicting the future motion of dynamic agents is of paramount importanc...

Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction

Trajectory prediction is a safety-critical tool for autonomous vehicles ...

You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction

Predicting the future trajectory of a moving agent can be easy when the ...

Please sign up or login with your details

Forgot password? Click here to reset