Cross-Domain Policy Adaptation via Value-Guided Data Filtering

by   Kang Xu, et al.

Generalizing policies across different domains with dynamics mismatch poses a significant challenge in reinforcement learning. For example, a robot learns the policy in a simulator, but when it is deployed in the real world, the dynamics of the environment may be different. Given the source and target domain with dynamics mismatch, we consider the online dynamics adaptation problem, in which case the agent can access sufficient source domain data while online interactions with the target domain are limited. Existing research has attempted to solve the problem from the dynamics discrepancy perspective. In this work, we reveal the limitations of these methods and explore the problem from the value difference perspective via a novel insight on the value consistency across domains. Specifically, we present the Value-Guided Data Filtering (VGDF) algorithm, which selectively shares transitions from the source domain based on the proximity of paired value targets across the two domains. Empirical results on various environments with kinematic and morphology shifts demonstrate that our method achieves superior performance compared to prior approaches.


Unsupervised Cross-domain Image Classification by Distance Metric Guided Feature Alignment

Learning deep neural networks that are generalizable across different do...

Focused Adaptation of Dynamics Models for Deformable Object Manipulation

In order to efficiently learn a dynamics model for a task in a new envir...

Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers

We propose a simple, practical, and intuitive approach for domain adapta...

AdAM: Few-Shot Image Generation via Adaptation-Aware Kernel Modulation

Few-shot image generation (FSIG) aims to learn to generate new and diver...

EPOpt: Learning Robust Neural Network Policies Using Model Ensembles

Sample complexity and safety are major challenges when learning policies...

Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learning

Unsupervised reinforcement learning aims to acquire skills without prior...

Toward Real-World Pathological Voice Detection

Voice disorders significantly undermine people's ability to speak in the...

Please sign up or login with your details

Forgot password? Click here to reset