DeepAI AI Chat
Log In Sign Up

Improved Exploring Starts by Kernel Density Estimation-Based State-Space Coverage Acceleration in Reinforcement Learning

by   Maximilian Schenke, et al.

Reinforcement learning (RL) is currently a popular research topic in control engineering and has the potential to make its way to industrial and commercial applications. Corresponding RL controllers are trained in direct interaction with the controlled system, rendering them data-driven and performance-oriented solutions. The best practice of exploring starts (ES) is used by default to support the learning process via randomly picked initial states. However, this method might deliver strongly biased results if the system's dynamic and constraints lead to unfavorable sample distributions in the state space (e.g., condensed sample accumulation in certain state-space areas). To overcome this issue, a kernel density estimation-based state-space coverage acceleration (DESSCA) is proposed, which improves the ES concept by prioritizing infrequently visited states for a more balanced coverage of the state space during training. Considered test scenarios are mountain car, cartpole and electric motor control environments. Using DQN and DDPG as exemplary RL algorithms, it can be shown that DESSCA is a simple yet effective algorithmic extension to the established ES approach.


page 1

page 4


Assessing and Accelerating Coverage in Deep Reinforcement Learning

Current deep reinforcement learning (DRL) algorithms utilize randomness ...

Approximating Martingale Process for Variance Reduction in Deep Reinforcement Learning with Large State Space

Approximating Martingale Process (AMP) is proven to be effective for var...

Count-Based Exploration in Feature Space for Reinforcement Learning

We introduce a new count-based optimistic exploration algorithm for Rein...

On the Unreasonable Efficiency of State Space Clustering in Personalization Tasks

In this effort we consider a reinforcement learning (RL) technique for s...

Reinforcement Learning for Traffic Signal Control: Comparison with Commercial Systems

Recently, Intelligent Transportation Systems are leveraging the power of...

Characterization of Human Balance through a Reinforcement Learning-based Muscle Controller

Balance assessment during physical rehabilitation often relies on rubric...