Safety Verification for Neural Networks Based on Set-boundary Analysis

by   Zhen Liang, et al.
National University of Defense Technology
State Key Laboratory of Computer Science, Institute

Neural networks (NNs) are increasingly applied in safety-critical systems such as autonomous vehicles. However, they are fragile and are often ill-behaved. Consequently, their behaviors should undergo rigorous guarantees before deployment in practice. In this paper we propose a set-boundary reachability method to investigate the safety verification problem of NNs from a topological perspective. Given an NN with an input set and a safe set, the safety verification problem is to determine whether all outputs of the NN resulting from the input set fall within the safe set. In our method, the homeomorphism property of NNs is mainly exploited, which establishes a relationship mapping boundaries to boundaries. The exploitation of this property facilitates reachability computations via extracting subsets of the input set rather than the entire input set, thus controlling the wrapping effect in reachability analysis and facilitating the reduction of computation burdens for safety verification. The homeomorphism property exists in some widely used NNs such as invertible NNs. Notable representations are invertible residual networks (i-ResNets) and Neural ordinary differential equations (Neural ODEs). For these NNs, our set-boundary reachability method only needs to perform reachability analysis on the boundary of the input set. For NNs which do not feature this property with respect to the input set, we explore subsets of the input set for establishing the local homeomorphism property, and then abandon these subsets for reachability computations. Finally, some examples demonstrate the performance of the proposed method.


Verifying Safety of Neural Networks from Topological Perspectives

Neural networks (NNs) are increasingly applied in safety-critical system...

Backward Reachability Analysis of Neural Feedback Loops: Techniques for Linear and Nonlinear Systems

The increasing prevalence of neural networks (NNs) in safety-critical ap...

Classification-based Approximate Reachability with Guarantees Applied to Safe Trajectory Tracking

Hamilton-Jacobi (HJ) reachability analysis has been developed over the p...

Backward Reachability Analysis for Neural Feedback Loops

The increasing prevalence of neural networks (NNs) in safety-critical ap...

Data-Driven Assessment of Deep Neural Networks with Random Input Uncertainty

When using deep neural networks to operate safety-critical systems, asse...

On The Verification of Neural ODEs with Stochastic Guarantees

We show that Neural ODEs, an emerging class of time-continuous neural ne...

Case Study: Verifying the Safety of an Autonomous Racing Car with a Neural Network Controller

This paper describes a verification case study on an autonomous racing c...

Please sign up or login with your details

Forgot password? Click here to reset