Scalable Task-Driven Robotic Swarm Control via Collision Avoidance and Learning Mean-Field Control

by   Kai Cui, et al.

In recent years, reinforcement learning and its multi-agent analogue have achieved great success in solving various complex control problems. However, multi-agent reinforcement learning remains challenging both in its theoretical analysis and empirical design of algorithms, especially for large swarms of embodied robotic agents where a definitive toolchain remains part of active research. We use emerging state-of-the-art mean-field control techniques in order to convert many-agent swarm control into more classical single-agent control of distributions. This allows profiting from advances in single-agent reinforcement learning at the cost of assuming weak interaction between agents. As a result, the mean-field model is violated by the nature of real systems with embodied, physically colliding agents. Here, we combine collision avoidance and learning of mean-field control into a unified framework for tractably designing intelligent robotic swarm behavior. On the theoretical side, we provide novel approximation guarantees for both general mean-field control in continuous spaces and with collision avoidance. On the practical side, we show that our approach outperforms multi-agent reinforcement learning and allows for decentralized open-loop application while avoiding collisions, both in simulation and real UAV swarms. Overall, we propose a framework for the design of swarm behavior that is both mathematically well-founded and practically useful, enabling the solution of otherwise intractable swarm problems.


page 1

page 6


Graphon Mean-Field Control for Cooperative Multi-Agent Reinforcement Learning

The marriage between mean-field theory and reinforcement learning has sh...

Learning Decentralized Partially Observable Mean Field Control for Artificial Collective Behavior

Recent reinforcement learning (RL) methods have achieved success in vari...

Using Reinforcement Learning to Herd a Robotic Swarm to a Target Distribution

In this paper, we present a reinforcement learning approach to designing...

Emergent Escape-based Flocking Behavior using Multi-Agent Reinforcement Learning

In nature, flocking or swarm behavior is observed in many species as it ...

Learning Sparse Graphon Mean Field Games

Although the field of multi-agent reinforcement learning (MARL) has made...

Stigmergy-based collision-avoidance algorithm for self-organising swarms

Real-time multi-agent collision-avoidance algorithms comprise a key enab...

Inverse Reinforcement Learning in Swarm Systems

Inverse reinforcement learning (IRL) has become a useful tool for learni...

Please sign up or login with your details

Forgot password? Click here to reset