Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning

by   Guozheng Ma, et al.

Data augmentation (DA) is a crucial technique for enhancing the sample efficiency of visual reinforcement learning (RL) algorithms. Notably, employing simple observation transformations alone can yield outstanding performance without extra auxiliary representation tasks or pre-trained encoders. However, it remains unclear which attributes of DA account for its effectiveness in achieving sample-efficient visual RL. To investigate this issue and further explore the potential of DA, this work conducts comprehensive experiments to assess the impact of DA's attributes on its efficacy and provides the following insights and improvements: (1) For individual DA operations, we reveal that both ample spatial diversity and slight hardness are indispensable. Building on this finding, we introduce Random PadResize (Rand PR), a new DA operation that offers abundant spatial diversity with minimal hardness. (2) For multi-type DA fusion schemes, the increased DA hardness and unstable data distribution result in the current fusion schemes being unable to achieve higher sample efficiency than their corresponding individual operations. Taking the non-stationary nature of RL into account, we propose a RL-tailored multi-type DA fusion scheme called Cycling Augmentation (CycAug), which performs periodic cycles of different DA operations to increase type diversity while maintaining data distribution consistency. Extensive evaluations on the DeepMind Control suite and CARLA driving simulator demonstrate that our methods achieve superior sample efficiency compared with the prior state-of-the-art methods.


page 5

page 6

page 7

page 15

page 27


A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning

Visual reinforcement learning (RL), which makes decisions directly from ...

DADA: Differentiable Automatic Data Augmentation

Data augmentation (DA) techniques aim to increase data variability, and ...

Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning

One of the key challenges in visual Reinforcement Learning (RL) is to le...

EPiDA: An Easy Plug-in Data Augmentation Framework for High Performance Text Classification

Recent works have empirically shown the effectiveness of data augmentati...

Reinforcement learning and Bayesian data assimilation for model-informed precision dosing in oncology

Model-informed precision dosing (MIPD) using therapeutic drug/biomarker ...

Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires

Quality-Diversity (QD) algorithms are powerful exploration algorithms th...

Please sign up or login with your details

Forgot password? Click here to reset