Derivative-Free Reinforcement Learning: A Review

by   Hong Qian, et al.

Reinforcement learning is about learning agent models that make the best sequential decisions in unknown environments. In an unknown environment, the agent needs to explore the environment while exploiting the collected information, which usually forms a sophisticated problem to solve. Derivative-free optimization, meanwhile, is capable of solving sophisticated problems. It commonly uses a sampling-and-updating framework to iteratively improve the solution, where exploration and exploitation are also needed to be well balanced. Therefore, derivative-free optimization deals with a similar core issue as reinforcement learning, and has been introduced in reinforcement learning approaches, under the names of learning classifier systems and neuroevolution/evolutionary reinforcement learning. Although such methods have been developed for decades, recently, derivative-free reinforcement learning exhibits attracting increasing attention. However, recent survey on this topic is still lacking. In this article, we summarize methods of derivative-free reinforcement learning to date, and organize the methods in aspects including parameter updating, model selection, exploration, and parallel/distributed methods. Moreover, we discuss some current limitations and possible future directions, hoping that this article could bring more attentions to this topic and serve as a catalyst for developing novel and efficient approaches.


page 1

page 2

page 3

page 4


A Survey of Exploration Methods in Reinforcement Learning

Exploration is an essential component of reinforcement learning algorith...

A Short Survey on Probabilistic Reinforcement Learning

A reinforcement learning agent tries to maximize its cumulative payoff b...

Reinforcement Learning Approaches in Social Robotics

There is a growing body of literature that formulates social human-robot...

A Survey of Reinforcement Learning Techniques: Strategies, Recent Development, and Future Directions

Reinforcement learning is one of the core components in designing an art...

ZOOpt/ZOOjl: Toolbox for Derivative-Free Optimization

Recent advances of derivative-free optimization allow efficient approxim...

Model-Based Reinforcement Learning for Physical Systems Without Velocity and Acceleration Measurements

In this paper, we propose a derivative-free model learning framework for...

Please sign up or login with your details

Forgot password? Click here to reset