Effects of Different Optimization Formulations in Evolutionary Reinforcement Learning on Diverse Behavior Generation

by   Victor Villin, et al.
Osaka Prefecture University

Generating various strategies for a given task is challenging. However, it has already proven to bring many assets to the main learning process, such as improved behavior exploration. With the growth in the interest of heterogeneity in solution in evolutionary computation and reinforcement learning, many promising approaches have emerged. To better understand how one guides multiple policies toward distinct strategies and benefit from diversity, we need to analyze further the influence of the reward signal modulation and other evolutionary mechanisms on the obtained behaviors. To that effect, this paper considers an existing evolutionary reinforcement learning framework which exploits multi-objective optimization as a way to obtain policies that succeed at behavior-related tasks as well as completing the main goal. Experiments on the Atari games stress that optimization formulations which do not consider objectives equally fail at generating diversity and even output agents that are worse at solving the problem at hand, regardless of the obtained behaviors.


Influencing Long-Term Behavior in Multiagent Reinforcement Learning

The main challenge of multiagent reinforcement learning is the difficult...

Diverse Policies Converge in Reward-free Markov Decision Processe

Reinforcement learning has achieved great success in many decision-makin...

Effective Diversity in Population-Based Reinforcement Learning

Maintaining a population of solutions has been shown to increase explora...

Behavior-based Neuroevolutionary Training in Reinforcement Learning

In addition to their undisputed success in solving classical optimizatio...

A Multiobjective Reinforcement Learning Framework for Microgrid Energy Management

The emergence of microgrids (MGs) has provided a promising solution for ...

The Quality-Diversity Transformer: Generating Behavior-Conditioned Trajectories with Decision Transformers

In the context of neuroevolution, Quality-Diversity algorithms have prov...

Exploring Novel Quality Diversity Methods For Generalization in Reinforcement Learning

The Reinforcement Learning field is strong on achievements and weak on r...

Please sign up or login with your details

Forgot password? Click here to reset