Towards Evaluating and Training Verifiably Robust Neural Networks

by   Zhaoyang Lyu, et al.
The Chinese University of Hong Kong

Recent works have shown that interval bound propagation (IBP) can be used to train verifiably robust neural networks. Reseachers observe an intriguing phenomenon on these IBP trained networks: CROWN, a bounding method based on tight linear relaxation, often gives very loose bounds on these networks. We also observe that most neurons become dead during the IBP training process, which could hurt the representation capability of the network. In this paper, we study the relationship between IBP and CROWN, and prove that CROWN is always tighter than IBP when choosing appropriate bounding lines. We further propose a relaxed version of CROWN, linear bound propagation (LBP), that can be used to verify large networks to obtain lower verified errors than IBP. We also design a new activation function, parameterized ramp function (ParamRamp), which has more diversity of neuron status than ReLU. We conduct extensive experiments on MNIST, CIFAR-10 and Tiny-ImageNet with ParamRamp activation and achieve state-of-the-art verified robustness. Code and the appendix are available at


page 1

page 2

page 3

page 4


Towards Stable and Efficient Training of Verifiably Robust Neural Networks

Training neural networks with verifiable robustness guarantees is challe...

On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models

Recent works have shown that it is possible to train models that are ver...

AP: Selective Activation for De-sparsifying Pruned Neural Networks

The rectified linear unit (ReLU) is a highly successful activation funct...

Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness

Certifiable robustness is a highly desirable property for adopting deep ...

Understanding Certified Training with Interval Bound Propagation

As robustness verification methods are becoming more precise, training c...

Automatic Perturbation Analysis on General Computational Graphs

Linear relaxation based perturbation analysis for neural networks, which...

Boosting Verified Training for Robust Image Classifications via Abstraction

This paper proposes a novel, abstraction-based, certified training metho...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset