On the Utility of Model Learning in HRI

by   Rohan Choudhury*, et al.

Fundamental to robotics is the debate between model-based and model-free learning: should the robot build an explicit model of the world, or learn a policy directly? In the context of HRI, part of the world to be modeled is the human. One option is for the robot to treat the human as a black box and learn a policy for how they act directly. But it can also model the human as an agent, and rely on a "theory of mind" to guide or bias the learning (grey box). We contribute a characterization of the performance of these methods under the optimistic case of having an ideal theory of mind, as well as under different scenarios in which the assumptions behind the robot's theory of mind for the human are wrong, as they inevitably will be in practice. We find that there is a significant sample complexity advantage to theory of mind methods and that they are more robust to covariate shift, but that when enough interaction data is available, black box approaches eventually dominate.


page 1

page 4

page 7

page 8


Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics

The most data-efficient algorithms for reinforcement learning in robotic...

Leolani: a reference machine with a theory of mind for social communication

Our state of mind is based on experiences and what other people tell us....

When Would You Trust a Robot? A Study on Trust and Theory of Mind in Human-Robot Interactions

Trust is a critical issue in Human Robot Interactions as it is the core ...

MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks

An ideal integration of autonomous agents in a human world implies that ...

Domain Knowledge Integration By Gradient Matching For Sample-Efficient Reinforcement Learning

Model-free deep reinforcement learning (RL) agents can learn an effectiv...

Model-free, Model-based, and General Intelligence

During the 60s and 70s, AI researchers explored intuitions about intelli...

Mind the hubris: complexity can misfire

Here we briefly reflect on the philosophical foundations that ground the...

Please sign up or login with your details

Forgot password? Click here to reset