Learning Altruistic Behaviours in Reinforcement Learning without External Rewards

by   Tim Franzmeyer, et al.

Can artificial agents learn to assist others in achieving their goals without knowing what those goals are? Generic reinforcement learning agents could be trained to behave altruistically towards others by rewarding them for altruistic behaviour, i.e., rewarding them for benefiting other agents in a given situation. Such an approach assumes that other agents' goals are known so that the altruistic agent can cooperate in achieving those goals. However, explicit knowledge of other agents' goals is often difficult to acquire. In the case of human agents, their goals and preferences may be difficult to express fully, may be ambiguous or even contradictory. Thus, it is beneficial to develop agents that do not depend on external supervision and can learn altruistic behaviour in a task-agnostic manner. We propose to act altruistically towards other agents by giving them more choice and thereby allowing them to better achieve their goals. Some concrete examples include opening a door for others or safeguarding them to pursue their objectives without interference. We formalize this concept and propose an altruistic agent that learns to increase the choices another agent has by preferring to maximize the number of states that the other agent can reach in its future. We evaluate our approach on three different multi-agent environments where another agent's success depends on the altruistic agent's behaviour. Finally, we show that our unsupervised agents can perform comparably to agents explicitly trained to work cooperatively, in some cases even outperforming them.


Modeling Others using Oneself in Multi-Agent Reinforcement Learning

We consider the multi-agent reinforcement learning setting with imperfec...

Coupling purposes with status-functions in artificial institutions

In multi-agent systems, the agents may have goals that depend on a socia...

Unsupervised Doodling and Painting with Improved SPIRAL

We investigate using reinforcement learning agents as generative models ...

A Formal Framework of Virtual Organisations as Agent Societies

We propose a formal framework that supports a model of agent-based Virtu...

Learning to Visually Navigate in Photorealistic Environments Without any Supervision

Learning to navigate in a realistic setting where an agent must rely sol...

Do deep reinforcement learning agents model intentions?

Inferring other agents' mental states such as their knowledge, beliefs a...

Prototyping three key properties of specific curiosity in computational reinforcement learning

Curiosity for machine agents has been a focus of intense research. The s...

Please sign up or login with your details

Forgot password? Click here to reset