Identifying Label Errors in Object Detection Datasets by Loss Inspection

by   Marius Schubert, et al.

Labeling datasets for supervised object detection is a dull and time-consuming task. Errors can be easily introduced during annotation and overlooked during review, yielding inaccurate benchmarks and performance degradation of deep neural networks trained on noisy labels. In this work, we for the first time introduce a benchmark for label error detection methods on object detection datasets as well as a label error detection method and a number of baselines. We simulate four different types of randomly introduced label errors on train and test sets of well-labeled object detection datasets. For our label error detection method we assume a two-stage object detector to be given and consider the sum of both stages' classification and regression losses. The losses are computed with respect to the predictions and the noisy labels including simulated label errors, aiming at detecting the latter. We compare our method to three baselines: a naive one without deep learning, the object detector's score and the entropy of the classification softmax distribution. We outperform all baselines and demonstrate that among the considered methods, ours is the only one that detects label errors of all four types efficiently. Furthermore, we detect real label errors a) on commonly used test datasets in object detection and b) on a proprietary dataset. In both cases we achieve low false positives rates, i.e., when considering 200 proposals from our method, we detect label errors with a precision for a) of up to 71.5


page 1

page 3

page 4

page 7

page 11

page 13


ObjectLab: Automated Diagnosis of Mislabeled Images in Object Detection Data

Despite powering sensitive systems like autonomous vehicles, object dete...

Effect of Annotation Errors on Drone Detection with YOLOv3

Following the recent advances in deep networks, object detection and tra...

Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks

We algorithmically identify label errors in the test sets of 10 of the m...

Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantification

In this work, we for the first time present a method for detecting label...

Object Detection with a Unified Label Space from Multiple Datasets

Given multiple datasets with different label spaces, the goal of this wo...

Training Object Detectors With Noisy Data

The availability of a large quantity of labelled training data is crucia...

Detecting Extratropical Cyclones of the Northern Hemisphere with Single Shot Detector

In this paper, we propose a deep learning-based model to detect extratro...

Please sign up or login with your details

Forgot password? Click here to reset