For SALE: State-Action Representation Learning for Deep Reinforcement Learning

by   Scott Fujimoto, et al.

In the field of reinforcement learning (RL), representation learning is a proven tool for complex image-based tasks, but is often overlooked for environments with low-level states, such as physical control problems. This paper introduces SALE, a novel approach for learning embeddings that model the nuanced interaction between state and action, enabling effective representation learning from low-level states. We extensively study the design space of these embeddings and highlight important design considerations. We integrate SALE and an adaptation of checkpoints for RL into TD3 to form the TD7 algorithm, which significantly outperforms existing continuous control algorithms. On OpenAI gym benchmark tasks, TD7 has an average performance gain of 276.7 TD3 at 300k and 5M time steps, respectively, and works in both the online and offline settings.


page 5

page 19

page 22

page 23

page 25

page 26

page 32

page 38


Dynamics-aware Embeddings

In this paper we consider self-supervised representation learning to imp...

Return-Based Contrastive Representation Learning for Reinforcement Learning

Recently, various auxiliary tasks have been proposed to accelerate repre...

Continuous Episodic Control

Non-parametric episodic memory can be used to quickly latch onto high-re...

Value-Consistent Representation Learning for Data-Efficient Reinforcement Learning

Deep reinforcement learning (RL) algorithms suffer severe performance de...

Measuring and Characterizing Generalization in Deep Reinforcement Learning

Deep reinforcement-learning methods have achieved remarkable performance...

Efficient Hierarchical Exploration with Stable Subgoal Representation Learning

Goal-conditioned hierarchical reinforcement learning (HRL) serves as a s...

Deep Tile Coder: an Efficient Sparse Representation Learning Approach with applications in Reinforcement Learning

Representation learning is critical to the success of modern large-scale...

Please sign up or login with your details

Forgot password? Click here to reset