Analysing Congestion Problems in Multi-agent Reinforcement Learning

by   Roxana Rădulescu, et al.

Congestion problems are omnipresent in today's complex networks and represent a challenge in many research domains. In the context of Multi-agent Reinforcement Learning (MARL), approaches like difference rewards and resource abstraction have shown promising results in tackling such problems. Resource abstraction was shown to be an ideal candidate for solving large-scale resource allocation problems in a fully decentralized manner. However, its performance and applicability strongly depends on some, until now, undocumented assumptions. Two of the main congestion benchmark problems considered in the literature are: the Beach Problem Domain and the Traffic Lane Domain. In both settings the highest system utility is achieved when overcrowding one resource and keeping the rest at optimum capacity. We analyse how abstract grouping can promote this behaviour and how feasible it is to apply this approach in a real-world domain (i.e., what assumptions need to be satisfied and what knowledge is necessary). We introduce a new test problem, the Road Network Domain (RND), where the resources are no longer independent, but rather part of a network (e.g., road network), thus choosing one path will also impact the load on other paths having common road segments. We demonstrate the application of state-of-the-art MARL methods for this new congestion model and analyse their performance. RND allows us to highlight an important limitation of resource abstraction and show that the difference rewards approach manages to better capture and inform the agents about the dynamics of the environment.


page 1

page 2

page 3

page 4


Resource Abstraction for Reinforcement Learning in Multiagent Congestion Problems

Real-world congestion problems (e.g. traffic congestion) are typically v...

Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization

Traffic congestion in metropolitan areas is a world-wide problem that ca...

LCS-TF: Multi-Agent Deep Reinforcement Learning-Based Intelligent Lane-Change System for Improving Traffic Flow

Discretionary lane-change is one of the critical challenges for autonomo...

Real-time and Large-scale Fleet Allocation of Autonomous Taxis: A Case Study in New York Manhattan Island

Nowadays, autonomous taxis become a highly promising transportation mode...

Deep Reinforcement Learning to Maximize Arterial Usage during Extreme Congestion

Collisions, crashes, and other incidents on road networks, if left unmit...

Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks

This paper presents a problem in power networks that creates an exciting...

r-Extreme Signalling for Congestion Control

In many "smart city" applications, congestion arises in part due to the ...

Please sign up or login with your details

Forgot password? Click here to reset