GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning

by   Xinshuo Weng, et al.

3D Multi-object tracking (MOT) is crucial to autonomous systems. Recent work uses a standard tracking-by-detection pipeline, where feature extraction is first performed independently for each object in order to compute an affinity matrix. Then the affinity matrix is passed to the Hungarian algorithm for data association. A key process of this standard pipeline is to learn discriminative features for different objects in order to reduce confusion during data association. In this work, we propose two techniques to improve the discriminative feature learning for MOT: (1) instead of obtaining features for each object independently, we propose a novel feature interaction mechanism by introducing the Graph Neural Network. As a result, the feature of one object is informed of the features of other objects so that the object feature can lean towards the object with similar feature (i.e., object probably with a same ID) and deviate from objects with dissimilar features (i.e., object probably with different IDs), leading to a more discriminative feature for each object; (2) instead of obtaining the feature from either 2D or 3D space in prior work, we propose a novel joint feature extractor to learn appearance and motion features from 2D and 3D space simultaneously. As features from different modalities often have complementary information, the joint feature can be more discriminate than feature from each individual modality. To ensure that the joint feature extractor does not heavily rely on one modality, we also propose an ensemble training paradigm. Through extensive evaluation, our proposed method achieves state-of-the-art performance on KITTI and nuScenes 3D MOT benchmarks. Our code will be made available at


page 4

page 8


Graph Neural Networks for 3D Multi-Object Tracking

3D Multi-object tracking (MOT) is crucial to autonomous systems. Recent ...

Relation3DMOT: Exploiting Deep Affinity for 3D Multi-Object Tracking from View Aggregation

Autonomous systems need to localize and track surrounding objects in 3D ...

Joint 3D Tracking and Forecasting with Graph Neural Network and Diversity Sampling

3D multi-object tracking (MOT) and trajectory forecasting are two critic...

Joint Detection and Multi-Object Tracking with Graph Neural Networks

Object detection and data association are critical components in multi-o...

Robust Multi-Modality Multi-Object Tracking

Multi-sensor perception is crucial to ensure the reliability and accurac...

A Unified Object Motion and Affinity Model for Online Multi-Object Tracking

Current popular online multi-object tracking (MOT) solutions apply singl...

Detection Recovery in Online Multi-Object Tracking with Sparse Graph Tracker

Joint object detection and online multi-object tracking (JDT) methods ha...

Please sign up or login with your details

Forgot password? Click here to reset