Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante

by   Alexis Duburcq, et al.

State-of-the-art reinforcement learning is now able to learn versatile locomotion, balancing and push-recovery capabilities for bipedal robots in simulation. Yet, the reality gap has mostly been overlooked and the simulated results hardly transfer to real hardware. Either it is unsuccessful in practice because the physics is over-simplified and hardware limitations are ignored, or regularity is not guaranteed and unexpected hazardous motions can occur. This paper presents a reinforcement learning framework capable of learning robust standing push recovery for bipedal robots with a smooth out-of-the-box transfer to reality, requiring only instantaneous proprioceptive observations. By combining original termination conditions and policy smoothness conditioning, we achieve stable learning, sim-to-real transfer and safety using a policy without memory nor observation history. Reward shaping is then used to give insights into how to keep balance. We demonstrate its performance in reality on the lower-limb medical exoskeleton Atalante.


page 1

page 2

page 6


On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning

Balancing and push-recovery are essential capabilities enabling humanoid...

Learning Whole-body Motor Skills for Humanoids

This paper presents a hierarchical framework for Deep Reinforcement Lear...

Sim-to-Real Learning of Robust Compliant Bipedal Locomotion on Torque Sensor-Less Gear-Driven Humanoid

In deep reinforcement learning, sim-to-real is the mainstream method as ...

Quadruped Capturability and Push Recovery via a Switched-Systems Characterization of Dynamic Balance

This paper studies capturability and push recovery for quadrupedal locom...

Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

This work aims to push the limits of agility for bipedal robots by enabl...

Learning to Improve Capture Steps for Disturbance Rejection in Humanoid Soccer

Over the past few years, soccer-playing humanoid robots have advanced si...

Sim-to-real transfer of active suspension control using deep reinforcement learning

We explore sim-to-real transfer of deep reinforcement learning controlle...

Please sign up or login with your details

Forgot password? Click here to reset