Reinforcement Learning with Prototypical Representations

by   Denis Yarats, et al.

Learning effective representations in image-based environments is crucial for sample efficient Reinforcement Learning (RL). Unfortunately, in RL, representation learning is confounded with the exploratory experience of the agent – learning a useful representation requires diverse data, while effective exploration is only possible with coherent representations. Furthermore, we would like to learn representations that not only generalize across tasks but also accelerate downstream exploration for efficient task-specific training. To address these challenges we propose Proto-RL, a self-supervised framework that ties representation learning with exploration through prototypical representations. These prototypes simultaneously serve as a summarization of the exploratory experience of an agent as well as a basis for representing observations. We pre-train these task-agnostic representations and prototypes on environments without downstream task information. This enables state-of-the-art downstream policy learning on a set of difficult continuous control tasks.


page 2

page 6

page 7

page 8

page 17

page 18


Exploratory State Representation Learning

Not having access to compact and meaningful representations is known to ...

Mask-based Latent Reconstruction for Reinforcement Learning

For deep reinforcement learning (RL) from pixels, learning effective sta...

Learning Actionable Representations with Goal-Conditioned Policies

Representation learning is a central challenge across a range of machine...

Learning Representations for Control with Hierarchical Forward Models

Learning control from pixels is difficult for reinforcement learning (RL...

Towards Learning Controllable Representations of Physical Systems

Learned representations of dynamical systems reduce dimensionality, pote...

Representations and Exploration for Deep Reinforcement Learning using Singular Value Decomposition

Representation learning and exploration are among the key challenges for...

Code Repositories


Proto-RL: Reinforcement Learning with Prototypical Representations

view repo


Implementation of ProtoRL with White-Noise

view repo

Please sign up or login with your details

Forgot password? Click here to reset