Phantom – A RL-driven multi-agent framework to model complex systems

by   Leo Ardon, et al.

Agent based modelling (ABM) is a computational approach to modelling complex systems by specifying the behaviour of autonomous decision-making components or agents in the system and allowing the system dynamics to emerge from their interactions. Recent advances in the field of Multi-agent reinforcement learning (MARL) have made it feasible to study the equilibrium of complex environments where multiple agents learn simultaneously. However, most ABM frameworks are not RL-native, in that they do not offer concepts and interfaces that are compatible with the use of MARL to learn agent behaviours. In this paper, we introduce a new open-source framework, Phantom, to bridge the gap between ABM and MARL. Phantom is an RL-driven framework for agent-based modelling of complex multi-agent systems including, but not limited to economic systems and markets. The framework aims to provide the tools to simplify the ABM specification in a MARL-compatible way - including features to encode dynamic partial observability, agent utility functions, heterogeneity in agent preferences or types, and constraints on the order in which agents can act (e.g. Stackelberg games, or more complex turn-taking environments). In this paper, we present these features, their design rationale and present two new environments leveraging the framework.


page 1

page 2

page 3

page 4


High Performance Simulation for Scalable Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning experiments and open-source training ...

The 4th International Workshop on Smart Simulation and Modelling for Complex Systems

Computer-based modelling and simulation have become useful tools to faci...

Tree of Knowledge: an Online Platform for Learning the Behaviour of Complex Systems

Many social sciences such as psychology and economics try to learn the b...

ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets

Model-free Reinforcement Learning (RL) requires the ability to sample tr...

Policy-focused Agent-based Modeling using RL Behavioral Models

Agent-based Models (ABMs) are valuable tools for policy analysis. ABMs h...

Sim-Env: Decoupling OpenAI Gym Environments from Simulation Models

Reinforcement learning (RL) is one of the most active fields of AI resea...

Discovering How Agents Learn Using Few Data

Decentralized learning algorithms are an essential tool for designing mu...

Please sign up or login with your details

Forgot password? Click here to reset