Explicitly Encouraging Low Fractional Dimensional Trajectories Via Reinforcement Learning

by   Sean Gillen, et al.

A key limitation in using various modern methods of machine learning in developing feedback control policies is the lack of appropriate methodologies to analyze their long-term dynamics, in terms of making any sort of guarantees (even statistically) about robustness. The central reasons for this are largely due to the so-called curse of dimensionality, combined with the black-box nature of the resulting control policies themselves. This paper aims at the first of these issues. Although the full state space of a system may be quite large in dimensionality, it is a common feature of most model-based control methods that the resulting closed-loop systems demonstrate dominant dynamics that are rapidly driven to some lower-dimensional sub-space within. In this work we argue that the dimensionality of this subspace is captured by tools from fractal geometry, namely various notions of a fractional dimension. We then show that the dimensionality of trajectories induced by model free reinforcement learning agents can be influenced adding a post processing function to the agents reward signal. We verify that the dimensionality reduction is robust to noise being added to the system and show that that the modified agents are more actually more robust to noise and push disturbances in general for the systems we examined.


page 1

page 2

page 3

page 4


Mesh Based Analysis of Low Fractal Dimension ReinforcementLearning Policies

In previous work, using a process we call meshing, the reachable state s...

Online Nonstochastic Model-Free Reinforcement Learning

In this work, we explore robust model-free reinforcement learning algori...

On the Search for Feedback in Reinforcement Learning

This paper addresses the problem of learning the optimal feedback policy...

Mesh-based Tools to Analyze Deep Reinforcement Learning Policies for Underactuated Biped Locomotion

In this paper, we present a mesh-based approach to analyze stability and...

Receding Horizon Inverse Reinforcement Learning

Inverse reinforcement learning (IRL) seeks to infer a cost function that...

Training in Task Space to Speed Up and Guide Reinforcement Learning

Recent breakthroughs in the reinforcement learning (RL) community have m...

Please sign up or login with your details

Forgot password? Click here to reset