Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks

by   Linrui zhang, et al.
Tsinghua University

Safety comes first in many real-world applications involving autonomous agents. Despite a large number of reinforcement learning (RL) methods focusing on safety-critical tasks, there is still a lack of high-quality evaluation of those algorithms that adheres to safety constraints at each decision step under complex and unknown dynamics. In this paper, we revisit prior work in this scope from the perspective of state-wise safe RL and categorize them as projection-based, recovery-based, and optimization-based approaches, respectively. Furthermore, we propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. This novel technique explicitly enforces hard constraints via the deep unrolling architecture and enjoys structural advantages in navigating the trade-off between reward improvement and constraint satisfaction. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit, a toolkit that provides off-the-shelf interfaces and evaluation utilities for safety-critical tasks. We then perform a comparative study of the involved algorithms on six benchmarks ranging from robotic control to autonomous driving. The empirical results provide an insight into their applicability and robustness in learning zero-cost-return policies without task-dependent handcrafting. The project page is available at https://sites.google.com/view/saferlkit.


page 11

page 14


State-wise Safe Reinforcement Learning: A Survey

Despite the tremendous success of Reinforcement Learning (RL) algorithms...

Approximate Model-Based Shielding for Safe Reinforcement Learning

Reinforcement learning (RL) has shown great potential for solving comple...

GUARD: A Safe Reinforcement Learning Benchmark

Due to the trial-and-error nature, it is typically challenging to apply ...

Adaptive Aggregation for Safety-Critical Control

Safety has been recognized as the central obstacle to preventing the use...

SafeRL-Kit: Evaluating Efficient Reinforcement Learning Methods for Safe Autonomous Driving

Safe reinforcement learning (RL) has achieved significant success on ris...

Learning to Play Table Tennis From Scratch using Muscular Robots

Dynamic tasks like table tennis are relatively easy to learn for humans ...

Prevalence of Code Smells in Reinforcement Learning Projects

Reinforcement Learning (RL) is being increasingly used to learn and adap...

Please sign up or login with your details

Forgot password? Click here to reset