Robust Feedback Motion Policy Design Using Reinforcement Learning on a 3D Digit Bipedal Robot

by   Guillermo A. Castillo, et al.

In this paper, a hierarchical and robust framework for learning bipedal locomotion is presented and successfully implemented on the 3D biped robot Digit built by Agility Robotics. We propose a cascade-structure controller that combines the learning process with intuitive feedback regulations. This design allows the framework to realize robust and stable walking with a reduced-dimension state and action spaces of the policy, significantly simplifying the design and reducing the sampling efficiency of the learning method. The inclusion of feedback regulation into the framework improves the robustness of the learned walking gait and ensures the success of the sim-to-real transfer of the proposed controller with minimal tuning. We specifically present a learning pipeline that considers hardware-feasible initial poses of the robot within the learning process to ensure the initial state of the learning is replicated as close as possible to the initial state of the robot in hardware experiments. Finally, we demonstrate the feasibility of our method by successfully transferring the learned policy in simulation to the Digit robot hardware, realizing sustained walking gaits under external force disturbances and challenging terrains not included during the training process. To the best of our knowledge, this is the first time a learning-based policy is transferred successfully to the Digit robot in hardware experiments without using dynamic randomization or curriculum learning.


page 1

page 4

page 5

page 7


Reinforcement Learning Meets Hybrid Zero Dynamics: A Case Study for RABBIT

The design of feedback controllers for bipedal robots is challenging due...

Iterative Reinforcement Learning Based Design of Dynamic Locomotion Skills for Cassie

Deep reinforcement learning (DRL) is a promising approach for developing...

Linear Policies are Sufficient to Realize Robust Bipedal Walking on Challenging Terrains

In this work, we demonstrate robust walking in the bipedal robot Digit o...

Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Biped

Learning controllers for bipedal robots is a challenging problem, often ...

Design, Simulation, and Testing of Laika, a Quadruped Robot with a Flexible Actuated Spine

Walking quadruped robots face challenges in positioning their feet and l...

Gait Library Synthesis for Quadruped Robots via Augmented Random Search

In this paper, with a view toward fast deployment of learned locomotion ...

Multiplicative Controller Fusion: A Hybrid Navigation Strategy For Deployment in Unknown Environments

Learning-based approaches often outperform hand-coded algorithmic soluti...

Please sign up or login with your details

Forgot password? Click here to reset