3D Adversarial Augmentations for Robust Out-of-Domain Predictions

by   Alexander Lehner, et al.

Since real-world training datasets cannot properly sample the long tail of the underlying data distribution, corner cases and rare out-of-domain samples can severely hinder the performance of state-of-the-art models. This problem becomes even more severe for dense tasks, such as 3D semantic segmentation, where points of non-standard objects can be confidently associated to the wrong class. In this work, we focus on improving the generalization to out-of-domain data. We achieve this by augmenting the training set with adversarial examples. First, we learn a set of vectors that deform the objects in an adversarial fashion. To prevent the adversarial examples from being too far from the existing data distribution, we preserve their plausibility through a series of constraints, ensuring sensor-awareness and shapes smoothness. Then, we perform adversarial augmentation by applying the learned sample-independent vectors to the available objects when training a model. We conduct extensive experiments across a variety of scenarios on data from KITTI, Waymo, and CrashD for 3D object detection, and on data from SemanticKITTI, Waymo, and nuScenes for 3D semantic segmentation. Despite training on a standard single dataset, our approach substantially improves the robustness and generalization of both 3D object detection and 3D semantic segmentation methods to out-of-domain data.


page 2

page 27

page 31


3D-VField: Learning to Adversarially Deform Point Clouds for Robust 3D Object Detection

As 3D object detection on point clouds relies on the geometrical relatio...

Adversarial Examples for Semantic Segmentation and Object Detection

It has been well demonstrated that adversarial examples, i.e., natural i...

Self-Regulation for Semantic Segmentation

In this paper, we seek reasons for the two major failure cases in Semant...

Real3D-Aug: Point Cloud Augmentation by Placing Real Objects with Occlusion Handling for 3D Detection and Segmentation

Object detection and semantic segmentation with the 3D lidar point cloud...

I Bet You Are Wrong: Gambling Adversarial Networks for Structured Semantic Segmentation

Adversarial training has been recently employed for realizing structured...

Rethinking Machine Learning Robustness via its Link with the Out-of-Distribution Problem

Despite multiple efforts made towards robust machine learning (ML) model...

Commonly Uncommon: Semantic Sparsity in Situation Recognition

Semantic sparsity is a common challenge in structured visual classificat...

Please sign up or login with your details

Forgot password? Click here to reset