Scaling MAP-Elites to Deep Neuroevolution

by   Cédric Colas, et al.

Quality-Diversity (QD) algorithms, and MAP-Elites (ME) in particular, have proven very useful for a broad range of applications including enabling real robots to recover quickly from joint damage, solving strongly deceptive maze tasks or evolving robot morphologies to discover new gaits. However, present implementations of MAP-Elites and other QD algorithms seem to be limited to low-dimensional controllers with far fewer parameters than modern deep neural network models. In this paper, we propose to leverage the efficiency of Evolution Strategies (ES) to scale MAP-Elites to high-dimensional controllers parameterized by large neural networks. We design and evaluate a new hybrid algorithm called MAP-Elites with Evolution Strategies (ME-ES) for post-damage recovery in a difficult high-dimensional control task where traditional ME fails. Additionally,we show that ME-ES performs efficient exploration, on par with state-of-the-art exploration algorithms in high-dimensional control tasks with strongly deceptive rewards.


page 6

page 7


Training Diverse High-Dimensional Controllers by Scaling Covariance Matrix Adaptation MAP-Annealing

Pre-training a diverse set of robot controllers in simulation has enable...

Challenges in High-dimensional Reinforcement Learning with Evolution Strategies

Evolution Strategies (ESs) have recently become popular for training dee...

Exploring Deep and Recurrent Architectures for Optimal Control

Sophisticated multilayer neural networks have achieved state of the art ...

Severe Damage Recovery in Evolving Soft Robots through Differentiable Programming

Biological systems are very robust to morphological damage, but artifici...

On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning

Balancing and push-recovery are essential capabilities enabling humanoid...

Intrinsic Motivation and Episodic Memories for Robot Exploration of High-Dimensional Sensory Spaces

This work presents an architecture that generates curiosity-driven goal-...

Code Repositories


Map-Elites based on Evolution Strategies

view repo

Please sign up or login with your details

Forgot password? Click here to reset