Human Engagement Providing Evaluative and Informative Advice for Interactive Reinforcement Learning

by   Adam Bignold, et al.

Reinforcement learning is an approach used by intelligent agents to autonomously learn new skills. Although reinforcement learning has been demonstrated to be an effective learning approach in several different contexts, a common drawback exhibited is the time needed in order to satisfactorily learn a task, especially in large state-action spaces. To address this issue, interactive reinforcement learning proposes the use of externally-sourced information in order to speed up the learning process. Up to now, different information sources have been used to give advice to the learner agent, among them human-sourced advice. When interacting with a learner agent, humans may provide either evaluative or informative advice. From the agent's perspective these styles of interaction are commonly referred to as reward-shaping and policy-shaping respectively. Evaluation requires the human to provide feedback on the prior action performed, while informative advice they provide advice on the best action to select for a given situation. Prior research has focused on the effect of human-sourced advice on the interactive reinforcement learning process, specifically aiming to improve the learning speed of the agent, while reducing the engagement with the human. This work presents an experimental setup for a human-trial designed to compare the methods people use to deliver advice in term of human engagement. Obtained results show that users giving informative advice to the learner agents provide more accurate advice, are willing to assist the learner agent for a longer time, and provide more advice per episode. Additionally, self-evaluation from participants using the informative approach has indicated that the agent's ability to follow the advice is higher, and therefore, they feel their own advice to be of higher accuracy when compared to people providing evaluative advice.


Deep Reinforcement Learning with Interactive Feedback in a Human-Robot Environment

Robots are extending their presence in domestic environments every day, ...

Improving interactive reinforcement learning: What makes a good teacher?

Interactive reinforcement learning has become an important apprenticeshi...

Learning to Engage with Interactive Systems: A field Study

Physical agents that can autonomously generate engaging, life-like behav...

Reinforcement learning with human advice. A survey

In this paper, we provide an overview of the existing methods for integr...

RLTIR: Activity-based Interactive Person Identification based on Reinforcement Learning Tree

Identity recognition plays an important role in ensuring security in our...

Persistent Rule-based Interactive Reinforcement Learning

Interactive reinforcement learning has allowed speeding up the learning ...

Please sign up or login with your details

Forgot password? Click here to reset