LASER: Learning a Latent Action Space for Efficient Reinforcement Learning

by   Arthur Allshire, et al.

The process of learning a manipulation task depends strongly on the action space used for exploration: posed in the incorrect action space, solving a task with reinforcement learning can be drastically inefficient. Additionally, similar tasks or instances of the same task family impose latent manifold constraints on the most effective action space: the task family can be best solved with actions in a manifold of the entire action space of the robot. Combining these insights we present LASER, a method to learn latent action spaces for efficient reinforcement learning. LASER factorizes the learning problem into two sub-problems, namely action space learning and policy learning in the new action space. It leverages data from similar manipulation task instances, either from an offline expert or online during policy learning, and learns from these trajectories a mapping from the original to a latent action space. LASER is trained as a variational encoder-decoder model to map raw actions into a disentangled latent action space while maintaining action reconstruction and latent space dynamic consistency. We evaluate LASER on two contact-rich robotic tasks in simulation, and analyze the benefit of policy learning in the generated latent action space. We show improved sample efficiency compared to the original action space from better alignment of the action space to the task space, as we observe with visualizations of the learned action space manifold. Additional details:


page 1

page 4

page 6


Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks

Reinforcement Learning (RL) of contact-rich manipulation tasks has yield...

PLAS: Latent Action Space for Offline Reinforcement Learning

The goal of offline reinforcement learning is to learn a policy from a f...

Learning Routines for Effective Off-Policy Reinforcement Learning

The performance of reinforcement learning depends upon designing an appr...

CLAS: Coordinating Multi-Robot Manipulation with Central Latent Action Spaces

Multi-robot manipulation tasks involve various control entities that can...

Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space

Intelligent agents must be able to think fast and slow to perform elabor...

DeFNet: Deconstructed Fabric Folding Strategy Based on Latent Space Roadmap and Flow-Based Policy

Fabric folding through robots is complex and challenging due to the defo...

Deep Episodic Memory: Encoding, Recalling, and Predicting Episodic Experiences for Robot Action Execution

We present a novel deep neural network architecture for representing rob...

Please sign up or login with your details

Forgot password? Click here to reset