Should Robots be Obedient?

by   Smitha Milli, et al.

Intuitively, obedience -- following the order that a human gives -- seems like a good property for a robot to have. But, we humans are not perfect and we may give orders that are not best aligned to our preferences. We show that when a human is not perfectly rational then a robot that tries to infer and act according to the human's underlying preferences can always perform better than a robot that simply follows the human's literal order. Thus, there is a tradeoff between the obedience of a robot and the value it can attain for its owner. We investigate how this tradeoff is impacted by the way the robot infers the human's preferences, showing that some methods err more on the side of obedience than others. We then analyze how performance degrades when the robot has a misspecified model of the features that the human cares about or the level of rationality of the human. Finally, we study how robots can start detecting such model misspecification. Overall, our work suggests that there might be a middle ground in which robots intelligently decide when to obey human orders, but err on the side of obedience.


page 1

page 2

page 3

page 4


Robots that Take Advantage of Human Trust

Humans often assume that robots are rational. We believe robots take opt...

Two ways to make your robot proactive: reasoning about human intentions, or reasoning about possible futures

Robots sharing their space with humans need to be proactive in order to ...

At Your Service: Coffee Beans Recommendation From a Robot Assistant

With advances in the field of machine learning, precisely algorithms for...

Including Uncertainty when Learning from Human Corrections

It is difficult for humans to efficiently teach robots how to correctly ...

Scaled Autonomy: Enabling Human Operators to Control Robot Fleets

Autonomous robots often encounter challenging situations where their con...

Morphology dictates a robot's ability to ground crowd-proposed language

As more robots act in physical proximity to people, it is essential to e...

Auditing Robot Learning for Safety and Compliance during Deployment

Robots of the future are going to exhibit increasingly human-like and su...

Please sign up or login with your details

Forgot password? Click here to reset