Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptation

by   Kang Xu, et al.

While Reinforcement Learning can achieve impressive results for complex tasks, the learned policies are generally prone to fail in downstream tasks with even minor model mismatch or unexpected perturbations. Recent works have demonstrated that a policy population with diverse behavior characteristics can generalize to downstream environments with various discrepancies. However, such policies might result in catastrophic damage during the deployment in practical scenarios like real-world systems due to the unrestricted behaviors of trained policies. Furthermore, training diverse policies without regulation of the behavior can result in inadequate feasible policies for extrapolating to a wide range of test conditions with dynamics shifts. In this work, we aim to train diverse policies under the regularization of the behavior patterns. We motivate our paradigm by observing the inverse dynamics in the environment with partial state information and propose Diversity in Regulation(DiR) training diverse policies with regulated behaviors to discover desired patterns that benefit the generalization. Considerable empirical results on various variations of different environments indicate that our method attains improvements over other diversity-driven counterparts.


page 12

page 15

page 16


One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL

While reinforcement learning algorithms can learn effective policies for...

Constructing a Good Behavior Basis for Transfer using Generalized Policy Updates

We study the problem of learning a good set of policies, so that when co...

The Quality-Diversity Transformer: Generating Behavior-Conditioned Trajectories with Decision Transformers

In the context of neuroevolution, Quality-Diversity algorithms have prov...

Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior

Learned locomotion policies can rapidly adapt to diverse environments si...

Learning a subspace of policies for online adaptation in Reinforcement Learning

Deep Reinforcement Learning (RL) is mainly studied in a setting where th...

Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalization

The proliferation of pretrained models, as a result of advancements in p...

Dynamics of niche construction in adaptable populations evolving in diverse environments

In both natural and artificial studies, evolution is often seen as synon...

Please sign up or login with your details

Forgot password? Click here to reset