A Surrogate-Assisted Controller for Expensive Evolutionary Reinforcement Learning

by   Yuxing Wang, et al.

The integration of Reinforcement Learning (RL) and Evolutionary Algorithms (EAs) aims at simultaneously exploiting the sample efficiency as well as the diversity and robustness of the two paradigms. Recently, hybrid learning frameworks based on this principle have achieved great success in various challenging robot control tasks. However, in these methods, policies from the genetic population are evaluated via interactions with the real environments, limiting their applicability in computationally expensive problems. In this work, we propose Surrogate-assisted Controller (SC), a novel and efficient module that can be integrated into existing frameworks to alleviate the computational burden of EAs by partially replacing the expensive policy evaluation. The key challenge in applying this module is to prevent the optimization process from being misled by the possible false minima introduced by the surrogate. To address this issue, we present two strategies for SC to control the workflow of hybrid frameworks. Experiments on six continuous control tasks from the OpenAI Gym platform show that SC can not only significantly reduce the cost of fitness evaluations, but also boost the performance of the original hybrid frameworks with collaborative learning and evolutionary processes.


page 4

page 7

page 8

page 9

page 10


Enabling surrogate-assisted evolutionary reinforcement learning via policy embedding

Evolutionary Reinforcement Learning (ERL) that applying Evolutionary Alg...

Algorithm Portfolio for Individual-based Surrogate-Assisted Evolutionary Algorithms

Surrogate-assisted evolutionary algorithms (SAEAs) are powerful optimisa...

Surrogate-assisted distributed swarm optimisation for computationally expensive models

Advances in parallel and distributed computing have enabled efficient im...

Surrogate Models for Enhancing the Efficiency of Neuroevolution in Reinforcement Learning

In the last years, reinforcement learning received a lot of attention. O...

Behavior-based Neuroevolutionary Training in Reinforcement Learning

In addition to their undisputed success in solving classical optimizatio...

Design Mining Interacting Wind Turbines

An initial study of surrogate-assisted evolutionary algorithms used to d...

Hybrid Indoor Localization via Reinforcement Learning-based Information Fusion

The paper is motivated by the importance of the Smart Cities (SC) concep...

Please sign up or login with your details

Forgot password? Click here to reset