DeepAI AI Chat
Log In Sign Up

Reachability Analysis of Neural Feedback Loops

by   Michael Everett, et al.

Neural Networks (NNs) can provide major empirical performance improvements for closed-loop systems, but they also introduce challenges in formally analyzing those systems' safety properties. In particular, this work focuses on estimating the forward reachable set of neural feedback loops (closed-loop systems with NN controllers). Recent work provides bounds on these reachable sets, but the computationally tractable approaches yield overly conservative bounds (thus cannot be used to verify useful properties), and the methods that yield tighter bounds are too intensive for online computation. This work bridges the gap by formulating a convex optimization problem for the reachability analysis of closed-loop systems with NN controllers. While the solutions are less tight than previous (semidefinite program-based) methods, they are substantially faster to compute, and some of those computational time savings can be used to refine the bounds through new input set partitioning techniques, which is shown to dramatically reduce the tightness gap. The new framework is developed for systems with uncertainty (e.g., measurement and process noise) and nonlinearities (e.g., polynomial dynamics), and thus is shown to be applicable to real-world systems. To inform the design of an initial state set when only the target state set is known/specified, a novel algorithm for backward reachability analysis is also provided, which computes the set of states that are guaranteed to lead to the target set. The numerical experiments show that our approach (based on linear relaxations and partitioning) gives a 5× reduction in conservatism in 150× less computation time compared to the state-of-the-art. Furthermore, experiments on quadrotor, 270-state, and polynomial systems demonstrate the method's ability to handle uncertainty sources, high dimensionality, and nonlinear dynamics, respectively.


page 1

page 15


Efficient Reachability Analysis of Closed-Loop Systems with Neural Network Controllers

Neural Networks (NNs) can provide major empirical performance improvemen...

Set based velocity shaping for robotic manipulators

We develop a new framework for trajectory planning on predefined paths, ...

A Hybrid Partitioning Strategy for Backward Reachability of Neural Feedback Loops

As neural networks become more integrated into the systems that we depen...

Neural Network Verification in Control

Learning-based methods could provide solutions to many of the long-stand...

Robustness Analysis of Neural Networks via Efficient Partitioning: Theory and Applications in Control Systems

Neural networks (NNs) are now routinely implemented on systems that must...

Efficient Backward Reachability Using the Minkowski Difference of Constrained Zonotopes

Backward reachability analysis is essential to synthesizing controllers ...

Code Repositories


Python tools for analyzing the robustness properties of neural networks (NNs) from MIT ACL

view repo