Annotating Motion Primitives for Simplifying Action Search in Reinforcement Learning

by   Isaac J. Sledge, et al.

Reinforcement learning in large-scale environments is challenging due to the many possible actions that can be taken in specific situations. We have previously developed a means of constraining, and hence speeding up, the search process through the use of motion primitives; motion primitives are sequences of pre-specified actions taken across a state series. As a byproduct of this work, we have found that if the motion primitives' motions and actions are labeled, then the search can be sped up further. Since motion primitives may initially lack such details, we propose a theoretically viewpoint-insensitive and speed-insensitive means of automatically annotating the underlying motions and actions. We do this through a differential-geometric, spatio-temporal kinematics descriptor, which analyzes how the poses of entities in two motion sequences change over time. We use this descriptor in conjunction with a weighted-nearest-neighbor classifier to label the primitives using a limited set of training examples. In our experiments, we achieve high motion and action annotation rates for human-action-derived primitives with as few as one training sample. We also demonstrate that reinforcement learning using accurately labeled trajectories leads to high-performing policies more quickly than standard reinforcement learning techniques. This is partly because motion primitives encode prior domain knowledge and preempt the need to re-discover that knowledge during training. It is also because agents can leverage the labels to systematically ignore action classes that do not facilitate task objectives, thereby reducing the action space.


page 2

page 3

page 19

page 20

page 21

page 22

page 23

page 25


Learning Topological Motion Primitives for Knot Planning

In this paper, we approach the challenging problem of motion planning fo...

Contact Mode Guided Sampling-Based Planning for Quasistatic Dexterous Manipulation in 2D

The discontinuities and multi-modality introduced by contacts make manip...

Animating an Autonomous 3D Talking Avatar

One of the main challenges with embodying a conversational agent is anno...

Hierarchical Learning with Unsupervised Skill Discovery for Highway Merging Applications

Driving in dense traffic with human and autonomous drivers is a challeng...

TEACH: Temporal Action Composition for 3D Humans

Given a series of natural language descriptions, our task is to generate...

Hierarchical Motion Understanding via Motion Programs

Current approaches to video analysis of human motion focus on raw pixels...

Quantizing Euclidean motions via double-coset decomposition

Concepts from mathematical crystallography and group theory are used her...

Please sign up or login with your details

Forgot password? Click here to reset