Improved Exploring Starts by Kernel Density Estimation-Based State-Space Coverage Acceleration in Reinforcement Learning

05/19/2021
by   Maximilian Schenke, et al.
0

Reinforcement learning (RL) is currently a popular research topic in control engineering and has the potential to make its way to industrial and commercial applications. Corresponding RL controllers are trained in direct interaction with the controlled system, rendering them data-driven and performance-oriented solutions. The best practice of exploring starts (ES) is used by default to support the learning process via randomly picked initial states. However, this method might deliver strongly biased results if the system's dynamic and constraints lead to unfavorable sample distributions in the state space (e.g., condensed sample accumulation in certain state-space areas). To overcome this issue, a kernel density estimation-based state-space coverage acceleration (DESSCA) is proposed, which improves the ES concept by prioritizing infrequently visited states for a more balanced coverage of the state space during training. Considered test scenarios are mountain car, cartpole and electric motor control environments. Using DQN and DDPG as exemplary RL algorithms, it can be shown that DESSCA is a simple yet effective algorithmic extension to the established ES approach.

READ FULL TEXT

page 1

page 4

research
12/01/2020

Assessing and Accelerating Coverage in Deep Reinforcement Learning

Current deep reinforcement learning (DRL) algorithms utilize randomness ...
research
11/29/2022

Approximating Martingale Process for Variance Reduction in Deep Reinforcement Learning with Large State Space

Approximating Martingale Process (AMP) is proven to be effective for var...
research
06/25/2017

Count-Based Exploration in Feature Space for Reinforcement Learning

We introduce a new count-based optimistic exploration algorithm for Rein...
research
12/24/2021

On the Unreasonable Efficiency of State Space Clustering in Personalization Tasks

In this effort we consider a reinforcement learning (RL) technique for s...
research
01/10/2023

Towards AI-controlled FES-restoration of arm movements: Controlling for progressive muscular fatigue with Gaussian state-space models

Reaching disability limits an individual's ability in performing daily t...
research
04/21/2021

Reinforcement Learning for Traffic Signal Control: Comparison with Commercial Systems

Recently, Intelligent Transportation Systems are leveraging the power of...
research
08/08/2023

Characterization of Human Balance through a Reinforcement Learning-based Muscle Controller

Balance assessment during physical rehabilitation often relies on rubric...

Please sign up or login with your details

Forgot password? Click here to reset