Reachability-based safe learning for optimal control problem

by   Stanislav Fedorov, et al.

In this work we seek for an approach to integrate safety in the learning process that relies on a partly known state-space model of the system and regards the unknown dynamics as an additive bounded disturbance. We introduce a framework for safely learning a control strategy for a given system with an additive disturbance. On the basis of the known part of the model, a safe set in which the system can learn safely, the algorithm can choose optimal actions for pursuing the target set as long as the safety-preserving condition is satisfied. After some learning episodes, the disturbance can be updated based on real-world data. To this end, Gaussian Process regression is conducted on the collected disturbance samples. Since the unstable nature of the law of the real world, for example, change of friction or conductivity with the temperature, we expect to have the more robust solution of optimal control problem. For evaluation of approach described above we choose an inverted pendulum as a benchmark model. The proposed algorithm manages to learn a policy that does not violate the pre-specified safety constraints. Observed performance is improved when it was incorporated exploration set up to make sure that an optimal policy is learned everywhere in the safe set. Finally, we outline some promising directions for future research beyond the scope of this paper.


Robust Regression for Safe Exploration in Control

We study the problem of safe learning and exploration in sequential cont...

Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions

Reinforcement Learning (RL) and continuous nonlinear control have been s...

Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems

Learning-based control algorithms require collection of abundant supervi...

Reachability Constrained Reinforcement Learning

Constrained reinforcement learning (CRL) has gained significant interest...

Safe Policy Search with Gaussian Process Models

We propose a method to optimise the parameters of a policy which will be...

Control Barriers in Bayesian Learning of System Dynamics

This paper focuses on learning a model of system dynamics online while s...

Learning Probabilistic Temporal Safety Properties from Examples in Relational Domains

We propose a framework for learning a fragment of probabilistic computat...

Please sign up or login with your details

Forgot password? Click here to reset