Guided Safe Shooting: model based reinforcement learning with safety constraints

by   Giuseppe Paolo, et al.

In the last decade, reinforcement learning successfully solved complex control tasks and decision-making problems, like the Go board game. Yet, there are few success stories when it comes to deploying those algorithms to real-world scenarios. One of the reasons is the lack of guarantees when dealing with and avoiding unsafe states, a fundamental requirement in critical control engineering systems. In this paper, we introduce Guided Safe Shooting (GuSS), a model-based RL approach that can learn to control systems with minimal violations of the safety constraints. The model is learned on the data collected during the operation of the system in an iterated batch fashion, and is then used to plan for the best action to perform at each time step. We propose three different safe planners, one based on a simple random shooting strategy and two based on MAP-Elites, a more advanced divergent-search algorithm. Experiments show that these planners help the learning agent avoid unsafe situations while maximally exploring the state space, a necessary aspect when learning an accurate model of the system. Furthermore, compared to model-free approaches, learning a model allows GuSS reducing the number of interactions with the real-system while still reaching high rewards, a fundamental requirement when handling engineering systems.


page 9

page 18

page 19

page 20

page 21


Safe Reinforcement Learning by Imagining the Near Future

Safe reinforcement learning is a promising path toward applying reinforc...

Modeling Survival in model-based Reinforcement Learning

Although recent model-free reinforcement learning algorithms have been s...

Safe Model-Free Reinforcement Learning using Disturbance-Observer-Based Control Barrier Functions

Safe reinforcement learning (RL) with assured satisfaction of hard state...

Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guarantees

Robustness and safety are critical for the trustworthy deployment of dee...

Online Constrained Model-based Reinforcement Learning

Applying reinforcement learning to robotic systems poses a number of cha...

Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

The last half-decade has seen a steep rise in the number of contribution...

Safe Control and Learning Using Generalized Action Governor

This paper introduces the Generalized Action Governor, which is a superv...

Please sign up or login with your details

Forgot password? Click here to reset