Multi-Agent Adversarial Inverse Reinforcement Learning

by   Lantao Yu, et al.

Reinforcement learning agents are prone to undesired behaviors due to reward mis-specification. Finding a set of reward functions to properly guide agent behaviors is particularly challenging in multi-agent scenarios. Inverse reinforcement learning provides a framework to automatically acquire suitable reward functions from expert demonstrations. Its extension to multi-agent settings, however, is difficult due to the more complex notions of rational behaviors. In this paper, we propose MA-AIRL, a new framework for multi-agent inverse reinforcement learning, which is effective and scalable for Markov games with high-dimensional state-action space and unknown dynamics. We derive our algorithm based on a new solution concept and maximum pseudolikelihood estimation within an adversarial reward learning framework. In the experiments, we demonstrate that MA-AIRL can recover reward functions that are highly correlated with ground truth ones, and significantly outperforms prior methods in terms of policy imitation.


page 1

page 2

page 3

page 4


Maximum Entropy Inverse Reinforcement Learning for Mean Field Games

Mean field games (MFG) facilitate the otherwise intractable reinforcemen...

Multi-Agent Generative Adversarial Imitation Learning

Imitation learning algorithms can be used to learn a policy from expert ...

Safety-Aware Multi-Agent Apprenticeship Learning

Our objective of this project is to make the extension based on the tech...

Towards Inverse Reinforcement Learning for Limit Order Book Dynamics

Multi-agent learning is a promising method to simulate aggregate competi...

Distributional Reward Estimation for Effective Multi-Agent Deep Reinforcement Learning

Multi-agent reinforcement learning has drawn increasing attention in pra...

Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behavior

In this work, we integrate `social' interactions into the MARL setup thr...

Please sign up or login with your details

Forgot password? Click here to reset