Assured Learning-enabled Autonomy: A Metacognitive Reinforcement Learning Framework

03/23/2021
by   Aquib Mustafa, et al.
0

Reinforcement learning (RL) agents with pre-specified reward functions cannot provide guaranteed safety across variety of circumstances that an uncertain system might encounter. To guarantee performance while assuring satisfaction of safety constraints across variety of circumstances, an assured autonomous control framework is presented in this paper by empowering RL algorithms with metacognitive learning capabilities. More specifically, adapting the reward function parameters of the RL agent is performed in a metacognitive decision-making layer to assure the feasibility of RL agent. That is, to assure that the learned policy by the RL agent satisfies safety constraints specified by signal temporal logic while achieving as much performance as possible. The metacognitive layer monitors any possible future safety violation under the actions of the RL agent and employs a higher-layer Bayesian RL algorithm to proactively adapt the reward function for the lower-layer RL agent. To minimize the higher-layer Bayesian RL intervention, a fitness function is leveraged by the metacognitive layer as a metric to evaluate success of the lower-layer RL agent in satisfaction of safety and liveness specifications, and the higher-layer Bayesian RL intervenes only if there is a risk of lower-layer RL failure. Finally, a simulation example is provided to validate the effectiveness of the proposed approach.

READ FULL TEXT

page 1

page 11

07/21/2021

Reinforcement Learning Agent Training with Goals for Real World Tasks

Reinforcement Learning (RL) is a promising approach for solving various ...
11/19/2020

Inverse Constrained Reinforcement Learning

Standard reinforcement learning (RL) algorithms train agents to maximize...
08/03/2023

Aligning Agent Policy with Externalities: Reward Design via Bilevel RL

In reinforcement learning (RL), a reward function is often assumed at th...
02/21/2023

Conditioning Hierarchical Reinforcement Learning on Flexible Constraints

Safety in goal directed Reinforcement Learning (RL) settings has typical...
03/19/2023

CLIP4MC: An RL-Friendly Vision-Language Model for Minecraft

One of the essential missions in the AI research community is to build a...
06/12/2020

Safety-guaranteed Reinforcement Learning based on Multi-class Support Vector Machine

Several works have addressed the problem of incorporating constraints in...
12/02/2022

STL-Based Synthesis of Feedback Controllers Using Reinforcement Learning

Deep Reinforcement Learning (DRL) has the potential to be used for synth...

Please sign up or login with your details

Forgot password? Click here to reset