Self-Imitation Learning of Locomotion Movements through Termination Curriculum

by   Amin Babadi, et al.

Animation and machine learning research have shown great advancements in the past decade, leading to robust and powerful methods for learning complex physically-based animations. However, learning can take hours or days, especially if no reference movement data is available. In this paper, we propose and evaluate a novel combination of techniques for accelerating the learning of stable locomotion movements through self-imitation learning of synthetic animations. First, we produce synthetic and cyclic reference movement using a recent online tree search approach that can discover stable walking gaits in a few minutes. This allows us to use reinforcement learning with Reference State Initialization (RSI) to find a neural network controller for imitating the synthesized reference motion. We further accelerate the learning using a novel curriculum learning approach called Termination Curriculum (TC), that adapts the episode termination threshold over time. The combination of the RSI and TC ensures that simulation budget is not wasted in regions of the state space not visited by the final policy. As a result, our agents can learn locomotion skills in just a few hours on a modest 4-core computer. We demonstrate this by producing locomotion movements for a variety of characters.


page 1

page 6

page 7


ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills

Humans are highly adept at walking in environments with foot placement c...

Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systems

Many tasks in practice require the collaboration of multiple agents thro...

MimicBot: Combining Imitation and Reinforcement Learning to win in Bot Bowl

This paper describe an hybrid agent trained to play in Fantasy Football ...

Human Motion Control of Quadrupedal Robots using Deep Reinforcement Learning

A motion-based control interface promises flexible robot operations in d...

HandMime: Sign Language Fingerspelling Acquisition via Imitation Learning

Learning fine-grained movements is among the most challenging topics in ...

Wheelless Soft Robotic Snake Locomotion: Study on Sidewinding and Helical Rolling Gaits

Soft robotic snakes (SRSs) have a unique combination of continuous and c...

Please sign up or login with your details

Forgot password? Click here to reset