3D for Free: Crossmodal Transfer Learning using HD Maps

by   Benjamin Wilson, et al.

3D object detection is a core perceptual challenge for robotics and autonomous driving. However, the class-taxonomies in modern autonomous driving datasets are significantly smaller than many influential 2D detection datasets. In this work, we address the long-tail problem by leveraging both the large class-taxonomies of modern 2D datasets and the robustness of state-of-the-art 2D detection methods. We proceed to mine a large, unlabeled dataset of images and LiDAR, and estimate 3D object bounding cuboids, seeded from an off-the-shelf 2D instance segmentation model. Critically, we constrain this ill-posed 2D-to-3D mapping by using high-definition maps and object size priors. The result of the mining process is 3D cuboids with varying confidence. This mining process is itself a 3D object detector, although not especially accurate when evaluated as such. However, we then train a 3D object detection model on these cuboids, consistent with other recent observations in the deep learning literature, we find that the resulting model is fairly robust to the noisy supervision that our mining process provides. We mine a collection of 1151 unlabeled, multimodal driving logs from an autonomous vehicle and use the discovered objects to train a LiDAR-based object detector. We show that detector performance increases as we mine more unlabeled data. With our full, unlabeled dataset, our method performs competitively with fully supervised methods, even exceeding the performance for certain object categories, without any human 3D annotations.


page 1

page 6


Large-Scale Object Discovery and Detector Adaptation from Unlabeled Video

We explore object discovery and detector adaptation based on unlabeled v...

MapFusion: A General Framework for 3D Object Detection with HDMaps

3D object detection is a key perception component in autonomous driving....

Interactively Test Driving an Object Detector: Estimating Performance on Unlabeled Data

In this paper, we study the problem of `test-driving' a detector, i.e. a...

nuScenes: A multimodal dataset for autonomous driving

Robust detection and tracking of objects is crucial for the deployment o...

Real-Time And Robust 3D Object Detection with Roadside LiDARs

This work aims to address the challenges in autonomous driving by focusi...

Learning to Detect Mobile Objects from LiDAR Scans Without Labels

Current 3D object detectors for autonomous driving are almost entirely t...

Deep Active Learning for Efficient Training of a LiDAR 3D Object Detector

Training a deep object detector for autonomous driving requires a huge a...

Please sign up or login with your details

Forgot password? Click here to reset