Verifying Safety of Neural Networks from Topological Perspectives

by   Zhen Liang, et al.
National University of Defense Technology
State Key Laboratory of Computer Science, Institute

Neural networks (NNs) are increasingly applied in safety-critical systems such as autonomous vehicles. However, they are fragile and are often ill-behaved. Consequently, their behaviors should undergo rigorous guarantees before deployment in practice. In this paper, we propose a set-boundary reachability method to investigate the safety verification problem of NNs from a topological perspective. Given an NN with an input set and a safe set, the safety verification problem is to determine whether all outputs of the NN resulting from the input set fall within the safe set. In our method, the homeomorphism property and the open map property of NNs are mainly exploited, which establish rigorous guarantees between the boundaries of the input set and the boundaries of the output set. The exploitation of these two properties facilitates reachability computations via extracting subsets of the input set rather than the entire input set, thus controlling the wrapping effect in reachability analysis and facilitating the reduction of computation burdens for safety verification. The homeomorphism property exists in some widely used NNs such as invertible residual networks (i-ResNets) and Neural ordinary differential equations (Neural ODEs), and the open map is a less strict property and easier to satisfy compared with the homeomorphism property. For NNs establishing either of these properties, our set-boundary reachability method only needs to perform reachability analysis on the boundary of the input set. Moreover, for NNs that do not feature these properties with respect to the input set, we explore subsets of the input set for establishing the local homeomorphism property and then abandon these subsets for reachability computations. Finally, some examples demonstrate the performance of the proposed method.


page 16

page 17

page 18

page 20

page 21


Safety Verification for Neural Networks Based on Set-boundary Analysis

Neural networks (NNs) are increasingly applied in safety-critical system...

Classification-based Approximate Reachability with Guarantees Applied to Safe Trajectory Tracking

Hamilton-Jacobi (HJ) reachability analysis has been developed over the p...

Backward Reachability Analysis of Neural Feedback Loops: Techniques for Linear and Nonlinear Systems

The increasing prevalence of neural networks (NNs) in safety-critical ap...

Backward Reachability Analysis for Neural Feedback Loops

The increasing prevalence of neural networks (NNs) in safety-critical ap...

Fast Falsification of Neural Networks using Property Directed Testing

Neural networks are now extensively used in perception, prediction and c...

Data-Driven Assessment of Deep Neural Networks with Random Input Uncertainty

When using deep neural networks to operate safety-critical systems, asse...

On The Verification of Neural ODEs with Stochastic Guarantees

We show that Neural ODEs, an emerging class of time-continuous neural ne...

Please sign up or login with your details

Forgot password? Click here to reset