3D-VField: Learning to Adversarially Deform Point Clouds for Robust 3D Object Detection

by   Alexander Lehner, et al.

As 3D object detection on point clouds relies on the geometrical relationships between the points, non-standard object shapes can hinder a method's detection capability. However, in safety-critical settings, robustness on out-of-distribution and long-tail samples is fundamental to circumvent dangerous issues, such as the misdetection of damaged or rare cars. In this work, we substantially improve the generalization of 3D object detectors to out-of-domain data by taking into account deformed point clouds during training. We achieve this with 3D-VField: a novel method that plausibly deforms objects via vectors learned in an adversarial fashion. Our approach constrains 3D points to slide along their sensor view rays while neither adding nor removing any of them. The obtained vectors are transferrable, sample-independent and preserve shape smoothness and occlusions. By augmenting normal samples with the deformations produced by these vector fields during training, we significantly improve robustness against differently shaped objects, such as damaged/deformed cars, even while training only on KITTI. Towards this end, we propose and share open source CrashD: a synthetic dataset of realistic damaged and rare cars, with a variety of crash scenarios. Extensive experiments on KITTI, Waymo, our CrashD and SUN RGB-D show the high generalizability of our techniques to out-of-domain data, different models and sensors, namely LiDAR and ToF cameras, for both indoor and outdoor scenes. Our CrashD dataset is available at https://crashd-cars.github.io.


page 8

page 9

page 10

page 11

page 12

page 16


3D Adversarial Augmentations for Robust Out-of-Domain Predictions

Since real-world training datasets cannot properly sample the long tail ...

3D Object Detection From LiDAR Data Using Distance Dependent Feature Extraction

This paper presents a new approach to 3D object detection that leverages...

Behind the Curtain: Learning Occluded Shapes for 3D Object Detection

Advances in LiDAR sensors provide rich 3D data that supports 3D scene un...

Frustum PointNets for 3D Object Detection from RGB-D Data

While object recognition on 2D images is getting more and more mature, 3...

DOPS: Learning to Detect 3D Objects and Predict their 3D Shapes

We propose DOPS, a fast single-stage 3D object detection method for LIDA...

TANet++: Triple Attention Network with Filtered Pointcloud on 3D Detection

TANet is one of state-of-the-art 3D object detection method on KITTI and...

Reassembling Broken Objects using Breaking Curves

Reassembling 3D broken objects is a challenging task. A robust solution ...

Please sign up or login with your details

Forgot password? Click here to reset