S^3Track: Self-supervised Tracking with Soft Assignment Flow

by   Fatemeh Azimi, et al.

In this work, we study self-supervised multiple object tracking without using any video-level association labels. We propose to cast the problem of multiple object tracking as learning the frame-wise associations between detections in consecutive frames. To this end, we propose differentiable soft object assignment for object association, making it possible to learn features tailored to object association with differentiable end-to-end training. With this training approach in hand, we develop an appearance-based model for learning instance-aware object features used to construct a cost matrix based on the pairwise distances between the object features. We train our model using temporal and multi-view data, where we obtain association pseudo-labels using optical flow and disparity information. Unlike most self-supervised tracking methods that rely on pretext tasks for learning the feature correspondences, our method is directly optimized for cross-object association in complex scenarios. As such, the proposed method offers a reidentification-based MOT approach that is robust to training hyperparameters and does not suffer from local minima, which are a challenge in self-supervised methods. We evaluate our proposed model on the KITTI, Waymo, nuScenes, and Argoverse datasets, consistently improving over other unsupervised methods (7.8% improvement in association accuracy on nuScenes).


page 2

page 7

page 11

page 12

page 13

page 14

page 15

page 16


CycAs: Self-supervised Cycle Association for Learning Re-identifiable Descriptions

This paper proposes a self-supervised learning method for the person re-...

Learning data association without data association: An EM approach to neural assignment prediction

Data association is a fundamental component of effective multi-object tr...

Object-Centric Multiple Object Tracking

Unsupervised object-centric learning methods allow the partitioning of s...

Tracking without Label: Unsupervised Multiple Object Tracking via Contrastive Similarity Learning

Unsupervised learning is a challenging task due to the lack of labels. M...

Deep Network Flow for Multi-Object Tracking

Data association problems are an important component of many computer vi...

MeNToS: Tracklets Association with a Space-Time Memory Network

We propose a method for multi-object tracking and segmentation (MOTS) th...

Self-Supervised Multi-Object Tracking with Cross-Input Consistency

In this paper, we propose a self-supervised learning procedure for train...

Please sign up or login with your details

Forgot password? Click here to reset