Towards Robust Sensor Fusion in Visual Perception

by   Shaojie Wang, et al.
Washington University in St Louis

We study the problem of robust sensor fusion in visual perception, especially under the autonomous driving settings. We evaluate the robustness of RGB camera and LiDAR sensor fusion for binary classification and object detection. In this work, we are interested in the behavior of different fusion methods under adversarial attacks on different sensors. We first train both classification and detection models with early fusion and late fusion, then apply different combinations of adversarial attacks on both sensor inputs for evaluation. We also study the effectiveness of adversarial attacks with varying budgets. Experiment results show that while sensor fusion models are generally vulnerable to adversarial attacks, late fusion method is more robust than early fusion. The results also provide insights on further obtaining robust sensor fusion models.


Sensor Adversarial Traits: Analyzing Robustness of 3D Object Detection Sensor Fusion Models

A critical aspect of autonomous vehicles (AVs) is the object detection s...

Fusion is Not Enough: Single-Modal Attacks to Compromise Fusion Models in Autonomous Driving

Multi-sensor fusion (MSF) is widely adopted for perception in autonomous...

Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving

Modern self-driving perception systems have been shown to improve upon p...

Scalable Primitives for Generalized Sensor Fusion in Autonomous Vehicles

In autonomous driving, there has been an explosion in the use of deep ne...

Adversarially robust deepfake media detection using fused convolutional neural network predictions

Deepfakes are synthetically generated images, videos or audios, which fr...

Adversarial Attacks on Camera-LiDAR Models for 3D Car Detection

Most autonomous vehicles (AVs) rely on LiDAR and RGB camera sensors for ...

End-to-end Uncertainty-based Mitigation of Adversarial Attacks to Automated Lane Centering

In the development of advanced driver-assistance systems (ADAS) and auto...

Please sign up or login with your details

Forgot password? Click here to reset