Formal Methods with a Touch of Magic

by   Parand Alizadeh Alamdari, et al.

Machine learning and formal methods have complimentary benefits and drawbacks. In this work, we address the controller-design problem with a combination of techniques from both fields. The use of black-box neural networks in deep reinforcement learning (deep RL) poses a challenge for such a combination. Instead of reasoning formally about the output of deep RL, which we call the wizard, we extract from it a decision-tree based model, which we refer to as the magic book. Using the extracted model as an intermediary, we are able to handle problems that are infeasible for either deep RL or formal methods by themselves. First, we suggest, for the first time, combining a magic book in a synthesis procedure. We synthesize a stand-alone correct-by-design controller that enjoys the favorable performance of RL. Second, we incorporate a magic book in a bounded model checking (BMC) procedure. BMC allows us to find numerous traces of the plant under the control of the wizard, which a user can use to increase the trustworthiness of the wizard and direct further training.


page 1

page 2

page 3

page 4


Deep Reinforcement Learning for Market Making Under a Hawkes Process-Based Limit Order Book Model

The stochastic control problem of optimal market making is among the cen...

Autonomous Blimp Control via H-infinity Robust Deep Residual Reinforcement Learning

Due to their superior energy efficiency, blimps may replace quadcopters ...

A Supervisory Control Algorithm Based on Property-Directed Reachability

We present an algorithm for synthesising a controller (supervisor) for a...

CEM-RL: Combining evolutionary and gradient-based methods for policy search

Deep neuroevolution and deep reinforcement learning (deep RL) algorithms...

Entropy Regularized Reinforcement Learning with Cascading Networks

Deep Reinforcement Learning (Deep RL) has had incredible achievements on...

Model-based gym environments for limit order book trading

Within the mathematical finance literature there is a rich catalogue of ...

Please sign up or login with your details

Forgot password? Click here to reset