Mean-Field Approximation of Cooperative Constrained Multi-Agent Reinforcement Learning (CMARL)

by   Washim Uddin Mondal, et al.

Mean-Field Control (MFC) has recently been proven to be a scalable tool to approximately solve large-scale multi-agent reinforcement learning (MARL) problems. However, these studies are typically limited to unconstrained cumulative reward maximization framework. In this paper, we show that one can use the MFC approach to approximate the MARL problem even in the presence of constraints. Specifically, we prove that, an N-agent constrained MARL problem, with state, and action spaces of each individual agents being of sizes |š’³|, and |š’°| respectively, can be approximated by an associated constrained MFC problem with an error, eā‰œš’Ŗ([āˆš(|š’³|)+āˆš(|š’°|)]/āˆš(N)). In a special case where the reward, cost, and state transition functions are independent of the action distribution of the population, we prove that the error can be improved to e=š’Ŗ(āˆš(|š’³|)/āˆš(N)). Also, we provide a Natural Policy Gradient based algorithm and prove that it can solve the constrained MARL problem within an error of š’Ŗ(e) with a sample complexity of š’Ŗ(e^-6).


page 1

page 2

page 3

page 4

āˆ™ 01/13/2023

Mean-Field Control based Approximation of Multi-Agent Reinforcement Learning in Presence of a Non-decomposable Shared Global State

Mean Field Control (MFC) is a powerful approximation tool to solve large...
āˆ™ 09/09/2021

On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)

Mean field control (MFC) is an effective way to mitigate the curse of di...
āˆ™ 02/28/2022

Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction?

Mean-Field Control (MFC) is a powerful tool to solve Multi-Agent Reinfor...
āˆ™ 11/10/2021

DeCOM: Decomposed Policy for Constrained Cooperative Multi-Agent Reinforcement Learning

In recent years, multi-agent reinforcement learning (MARL) has presented...
āˆ™ 02/26/2022

A Scalable Graph-Theoretic Distributed Framework for Cooperative Multi-Agent Reinforcement Learning

The main challenge of large-scale cooperative multi-agent reinforcement ...
āˆ™ 10/27/2021

Model based Multi-agent Reinforcement Learning with Tensor Decompositions

A challenge in multi-agent reinforcement learning is to be able to gener...
āˆ™ 06/29/2023

Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning

Many applications, e.g., in shared mobility, require coordinating a larg...

Please sign up or login with your details

Forgot password? Click here to reset