Bridging Images and Videos: A Simple Learning Framework for Large Vocabulary Video Object Detection

by   Sanghyun Woo, et al.

Scaling object taxonomies is one of the important steps toward a robust real-world deployment of recognition systems. We have faced remarkable progress in images since the introduction of the LVIS benchmark. To continue this success in videos, a new video benchmark, TAO, was recently presented. Given the recent encouraging results from both detection and tracking communities, we are interested in marrying those two advances and building a strong large vocabulary video tracker. However, supervisions in LVIS and TAO are inherently sparse or even missing, posing two new challenges for training the large vocabulary trackers. First, no tracking supervisions are in LVIS, which leads to inconsistent learning of detection (with LVIS and TAO) and tracking (only with TAO). Second, the detection supervisions in TAO are partial, which results in catastrophic forgetting of absent LVIS categories during video fine-tuning. To resolve these challenges, we present a simple but effective learning framework that takes full advantage of all available training data to learn detection and tracking while not losing any LVIS categories to recognize. With this new learning scheme, we show that consistent improvements of various large vocabulary trackers are capable, setting strong baseline results on the challenging TAO benchmarks.


page 1

page 2

page 3

page 4


TAO: A Large-Scale Benchmark for Tracking Any Object

For many years, multi-object tracking benchmarks have focused on a handf...

Detect or Track: Towards Cost-Effective Video Object Detection/Tracking

State-of-the-art object detectors and trackers are developing fast. Trac...

MV-YOLO: Motion Vector-aided Tracking by Semantic Object Detection

Object tracking is the cornerstone of many visual analytics systems. Whi...

Few-Shot Learning for Video Object Detection in a Transfer-Learning Scheme

Different from static images, videos contain additional temporal and spa...

V3Det: Vast Vocabulary Visual Detection Dataset

Recent advances in detecting arbitrary objects in the real world are tra...

Evaluation of Object Trackers in Distorted Surveillance Videos

Object tracking in realistic scenarios is a difficult problem affected b...

Please sign up or login with your details

Forgot password? Click here to reset