Improving Confidence in the Estimation of Values and Norms

Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent's behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.


page 1

page 2

page 3

page 4


Value Engineering for Autonomous Agents

Machine Ethics (ME) is concerned with the design of Artificial Moral Age...

Learning Norms from Stories: A Prior for Value Aligned Agents

Value alignment is a property of an intelligent agent indicating that it...

Engaging in Dialogue about an Agent's Norms and Behaviors

We present a set of capabilities allowing an agent planning with moral a...

Towards a computational model of social norms

We describe a computational model of social norms based on identifying v...

In conversation with Artificial Intelligence: aligning language models with human values

Large-scale language technologies are increasingly used in various forms...

Tipping Points for Norm Change in Human Cultures

Humans interact with each other on a daily basis by developing and maint...

Building Jiminy Cricket: An Architecture for Moral Agreements Among Stakeholders

An autonomous system is constructed by a manufacturer, operates in a soc...

Please sign up or login with your details

Forgot password? Click here to reset