Inverse Reinforcement Learning in Swarm Systems

by   Adrian Šošić, et al.

Inverse reinforcement learning (IRL) has become a useful tool for learning behavioral models from demonstration data. However, IRL remains mostly unexplored for multi-agent systems. In this paper, we show how the principle of IRL can be extended to homogeneous large-scale problems, inspired by the collective swarming behavior of natural systems. In particular, we make the following contributions to the field: 1) We introduce the swarMDP framework, a sub-class of decentralized partially observable Markov decision processes endowed with a swarm characterization. 2) Exploiting the inherent homogeneity of this framework, we reduce the resulting multi-agent IRL problem to a single-agent one by proving that the agent-specific value functions in this model coincide. 3) To solve the corresponding control problem, we propose a novel heterogeneous learning scheme that is particularly tailored to the swarm setting. Results on two example systems demonstrate that our framework is able to produce meaningful local reward models from which we can replicate the observed global system dynamics.


Learning Decentralized Partially Observable Mean Field Control for Artificial Collective Behavior

Recent reinforcement learning (RL) methods have achieved success in vari...

Scalable Task-Driven Robotic Swarm Control via Collision Avoidance and Learning Mean-Field Control

In recent years, reinforcement learning and its multi-agent analogue hav...

Swarm Modelling with Dynamic Mode Decomposition

Modelling biological or engineering swarms is challenging due to the inh...

Discovering Individual Rewards in Collective Behavior through Inverse Multi-Agent Reinforcement Learning

The discovery of individual objectives in collective behavior of complex...

Common Information based Approximate State Representations in Multi-Agent Reinforcement Learning

Due to information asymmetry, finding optimal policies for Decentralized...

Towards Safe and Efficient Swarm-Human Collaboration: A Hierarchical Multi-Agent Pickup and Delivery framework

The multi-Agent Pickup and Delivery (MAPD) problem is crucial in the rea...

Deep Reinforcement Learning for Swarm Systems

Recently, deep reinforcement learning (RL) methods have been applied suc...

Please sign up or login with your details

Forgot password? Click here to reset