Sim-to-Real Learning of Footstep-Constrained Bipedal Dynamic Walking

by   Helei Duan, et al.

Recently, work on reinforcement learning (RL) for bipedal robots has successfully learned controllers for a variety of dynamic gaits with robust sim-to-real demonstrations. In order to maintain balance, the learned controllers have full freedom of where to place the feet, resulting in highly robust gaits. In the real world however, the environment will often impose constraints on the feasible footstep locations, typically identified by perception systems. Unfortunately, most demonstrated RL controllers on bipedal robots do not allow for specifying and responding to such constraints. This missing control interface greatly limits the real-world application of current RL controllers. In this paper, we aim to maintain the robust and dynamic nature of learned gaits while also respecting footstep constraints imposed externally. We develop an RL formulation for training dynamic gait controllers that can respond to specified touchdown locations. We then successfully demonstrate simulation and sim-to-real performance on the bipedal robot Cassie. In addition, we use supervised learning to induce a transition model for accurately predicting the next touchdown locations that the controller can achieve given the robot's proprioceptive observations. This model paves the way for integrating the learned controller into a full-order robot locomotion planner that robustly satisfies both balance and environmental constraints.


page 1

page 4

page 6


Learning Dynamic Bipedal Walking Across Stepping Stones

In this work, we propose a learning approach for 3D dynamic bipedal walk...

Improving Input-Output Linearizing Controllers for Bipedal Robots via Reinforcement Learning

The main drawbacks of input-output linearizing controllers are the need ...

End-to-End Reinforcement Learning for Torque Based Variable Height Hopping

Legged locomotion is arguably the most suited and versatile mode to deal...

How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control?

The difficulty of optimal control problems has classically been characte...

Learning Bipedal Walking On Planned Footsteps For Humanoid Robots

Deep reinforcement learning (RL) based controllers for legged robots hav...

Deep Visual Perception for Dynamic Walking on Discrete Terrain

Dynamic bipedal walking on discrete terrain, like stepping stones, is a ...

Please sign up or login with your details

Forgot password? Click here to reset