On Safety Testing, Validation, and Characterization with Scenario-Sampling: A Case Study of Legged Robots

by   Bowen Weng, et al.

The dynamic response of the legged robot locomotion is non-Lipschitz and can be stochastic due to environmental uncertainties. To test, validate, and characterize the safety performance of legged robots, existing solutions on observed and inferred risk can be incomplete and sampling inefficient. Some formal verification methods suffer from the model precision and other surrogate assumptions. In this paper, we propose a scenario sampling based testing framework that characterizes the overall safety performance of a legged robot by specifying (i) where (in terms of a set of states) the robot is potentially safe, and (ii) how safe the robot is within the specified set. The framework can also help certify the commercial deployment of the legged robot in real-world environment along with human and compare safety performance among legged robots with different mechanical structures and dynamic properties. The proposed framework is further deployed to evaluate a group of state-of-the-art legged robot locomotion controllers from various model-based, deep neural network involved, and reinforcement learning based methods in the literature. Among a series of intended work domains of the studied legged robots (e.g. tracking speed on sloped surface, with abrupt changes on demanded velocity, and against adversarial push-over disturbances), we show that the method can adequately capture the overall safety characterization and the subtle performance insights. Many of the observed safety outcomes, to the best of our knowledge, have never been reported by the existing work in the legged robot literature.


page 1

page 6

page 7


Safe Reinforcement Learning for Legged Locomotion

Designing control policies for legged locomotion is complex due to the u...

A Formal Characterization of Black-Box System Safety Performance with Scenario Sampling

A typical scenario-based evaluation framework seeks to characterize a bl...

Rethink the Adversarial Scenario-based Safety Testing of Robots: the Comparability and Optimal Aggressiveness

This paper studies the class of scenario-based safety testing algorithms...

Modular Safety-Critical Control of Legged Robots

Safety concerns during the operation of legged robots must be addressed ...

Risk-Sensitive Path Planning via CVaR Barrier Functions: Application to Bipedal Locomotion

Enforcing safety of robotic systems in the presence of stochastic uncert...

Robot Motion Risk Reasoning Framework

This paper presents a formal and comprehensive reasoning framework for r...

Please sign up or login with your details

Forgot password? Click here to reset