Enabling surrogate-assisted evolutionary reinforcement learning via policy embedding

by   Lan Tang, et al.

Evolutionary Reinforcement Learning (ERL) that applying Evolutionary Algorithms (EAs) to optimize the weight parameters of Deep Neural Network (DNN) based policies has been widely regarded as an alternative to traditional reinforcement learning methods. However, the evaluation of the iteratively generated population usually requires a large amount of computational time and can be prohibitively expensive, which may potentially restrict the applicability of ERL. Surrogate is often used to reduce the computational burden of evaluation in EAs. Unfortunately, in ERL, each individual of policy usually represents millions of weights parameters of DNN. This high-dimensional representation of policy has introduced a great challenge to the application of surrogates into ERL to speed up training. This paper proposes a PE-SAERL Framework to at the first time enable surrogate-assisted evolutionary reinforcement learning via policy embedding (PE). Empirical results on 5 Atari games show that the proposed method can perform more efficiently than the four state-of-the-art algorithms. The training process is accelerated up to 7x on tested games, comparing to its counterpart without the surrogate and PE.


A Surrogate-Assisted Controller for Expensive Evolutionary Reinforcement Learning

The integration of Reinforcement Learning (RL) and Evolutionary Algorith...

Algorithm Portfolio for Individual-based Surrogate-Assisted Evolutionary Algorithms

Surrogate-assisted evolutionary algorithms (SAEAs) are powerful optimisa...

Behavior-based Neuroevolutionary Training in Reinforcement Learning

In addition to their undisputed success in solving classical optimizatio...

Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning prob...

Evolutionary Reinforcement Learning via Cooperative Coevolutionary Negatively Correlated Search

Evolutionary algorithms (EAs) have been successfully applied to optimize...

Voronoi-based Efficient Surrogate-assisted Evolutionary Algorithm for Very Expensive Problems

Very expensive problems are very common in practical system that one fit...

A Deep Neural Network Surrogate Modeling Benchmark for Temperature Field Prediction of Heat Source Layout

Thermal issue is of great importance during layout design of heat source...

Please sign up or login with your details

Forgot password? Click here to reset