A Novel Performance Evaluation Methodology for Single-Target Trackers

by   Matej Kristan, et al.

This paper addresses the problem of single-target tracker performance evaluation. We consider the performance measures, the dataset and the evaluation system to be the most important components of tracker evaluation and propose requirements for each of them. The requirements are the basis of a new evaluation methodology that aims at a simple and easily interpretable tracker comparison. The ranking-based methodology addresses tracker equivalence in terms of statistical significance and practical differences. A fully-annotated dataset with per-frame annotations with several visual attributes is introduced. The diversity of its visual properties is maximized in a novel way by clustering a large number of videos according to their visual attributes. This makes it the most sophistically constructed and annotated dataset to date. A multi-platform evaluation system allowing easy integration of third-party trackers is presented as well. The proposed evaluation methodology was tested on the VOT2014 challenge on the new dataset and 38 trackers, making it the largest benchmark to date. Most of the tested trackers are indeed state-of-the-art since they outperform the standard baselines, resulting in a highly-challenging benchmark. An exhaustive analysis of the dataset from the perspective of tracking difficulty is carried out. To facilitate tracker comparison a new performance visualization technique is proposed.


page 3

page 10

page 20

page 21


LSOTB-TIR:A Large-Scale High-Diversity Thermal Infrared Object Tracking Benchmark

In this paper, we present a Large-Scale and high-diversity general Therm...

Performance Evaluation Methodology for Long-Term Visual Object Tracking

A long-term visual object tracking performance evaluation methodology an...

Visual object tracking performance measures revisited

The problem of visual tracking evaluation is sporting a large variety of...

Person Monitoring by Full Body Tracking in Uniform Crowd Environment

Full body trackers are utilized for surveillance and security purposes, ...

PTB-TIR: A Thermal Infrared Pedestrian Tracking Benchmark

Thermal infrared (TIR) pedestrian tracking is one of the most important ...

Modular Decomposition and Analysis of Registration based Trackers

This paper presents a new way to study registration based trackers by de...

On some properties of medians, percentiles, baselines, and thresholds in empirical bibliometric analysis

One of the most useful and correct methodological approaches in bibliome...

Please sign up or login with your details

Forgot password? Click here to reset