Transformer-based assignment decision network for multiple object tracking

by   Athena Psalta, et al.

Data association is a crucial component for any multiple object tracking (MOT) method that follows the tracking-by-detection paradigm. To generate complete trajectories such methods employ a data association process to establish assignments between detections and existing targets during each timestep. Recent data association approaches try to solve a multi-dimensional linear assignment task or a network flow minimization problem or either tackle it via multiple hypotheses tracking. However, during inference an optimization step that computes optimal assignments is required for every sequence frame adding significant computational complexity in any given solution. To this end, in the context of this work we introduce Transformer-based Assignment Decision Network (TADN) that tackles data association without the need of any explicit optimization during inference. In particular, TADN can directly infer assignment pairs between detections and active targets in a single forward pass of the network. We have integrated TADN in a rather simple MOT framework, we designed a novel training strategy for efficient end-to-end training and demonstrate the high potential of our approach for online visual tracking-by-detection MOT on two popular benchmarks, i.e. MOT17 and UA-DETRAC. Our proposed approach outperforms the state-of-the-art in most evaluation metrics despite its simple nature as a tracker which lacks significant auxiliary components such as occlusion handling or re-identification. The implementation of our method is publicly available at


page 13

page 14


MOTR: End-to-End Multiple-Object Tracking with TRansformer

The key challenge in multiple-object tracking (MOT) task is temporal mod...

Learnable Graph Matching: Incorporating Graph Partitioning with Deep Feature Learning for Multiple Object Tracking

Data association across frames is at the core of Multiple Object Trackin...

SparseTrack: Multi-Object Tracking by Performing Scene Decomposition based on Pseudo-Depth

Exploring robust and efficient association methods has always been an im...

Learning data association without data association: An EM approach to neural assignment prediction

Data association is a fundamental component of effective multi-object tr...

3DMOTFormer: Graph Transformer for Online 3D Multi-Object Tracking

Tracking 3D objects accurately and consistently is crucial for autonomou...

Multi-object Tracking via End-to-end Tracklet Searching and Ranking

Recent works in multiple object tracking use sequence model to calculate...

MOTRv3: Release-Fetch Supervision for End-to-End Multi-Object Tracking

Although end-to-end multi-object trackers like MOTR enjoy the merits of ...

Please sign up or login with your details

Forgot password? Click here to reset