Robustness Certification of Visual Perception Models via Camera Motion Smoothing

by   Hanjiang Hu, et al.

A vast literature shows that the learning-based visual perception model is sensitive to adversarial noises but few works consider the robustness of robotic perception models under widely-existing camera motion perturbations. To this end, we study the robustness of the visual perception model under camera motion perturbations to investigate the influence of camera motion on robotic perception. Specifically, we propose a motion smoothing technique for arbitrary image classification models, whose robustness under camera motion perturbations could be certified. The proposed robustness certification framework based on camera motion smoothing provides tight and scalable robustness guarantees for visual perception modules so that they are applicable to wide robotic applications. As far as we are aware, this is the first work to provide the robustness certification for the deep perception module against camera motions, which improves the trustworthiness of robotic perception. A realistic indoor robotic dataset with the dense point cloud map for the entire room, MetaRoom, is introduced for the challenging certifiable robust perception task. We conduct extensive experiments to validate the certification approach via motion smoothing against camera motion perturbations. Our framework guarantees the certified accuracy of 81.7 direction within -0.1m ` 0.1m. We also validate the effectiveness of our method on the real-world robot by conducting hardware experiment on the robotic arm with an eye-in-hand camera. The code is available on


page 2

page 5

page 9

page 17

page 20


Certified Adversarial Robustness via Randomized Smoothing

Recent work has shown that any classifier which classifies well under Ga...

PourIt!: Weakly-supervised Liquid Perception from a Single Image for Visual Closed-Loop Robotic Pouring

Liquid perception is critical for robotic pouring tasks. It usually requ...

UniCR: Universally Approximated Certified Robustness via Randomized Smoothing

We study certified robustness of machine learning classifiers against ad...

Stitching Dynamic Movement Primitives and Image-based Visual Servo Control

Utilizing perception for feedback control in combination with Dynamic Mo...

Adversarially Robust Video Perception by Seeing Motion

Despite their excellent performance, state-of-the-art computer vision mo...

Illustrative Motion Smoothing for Attention Guidance in Dynamic Visualizations

3D animations are an effective method to learn about complex dynamic phe...

Skinned Motion Retargeting with Residual Perception of Motion Semantics Geometry

A good motion retargeting cannot be reached without reasonable considera...

Please sign up or login with your details

Forgot password? Click here to reset