CoLight: Learning Network-level Cooperation for Traffic Signal Control

by   Hua Wei, et al.

Cooperation is critical in multi-agent reinforcement learning (MARL). In the context of traffic signal control, good cooperation among the traffic signal agents enables the vehicles to move through intersections more smoothly. Conventional transportation approaches implement cooperation by pre-calculating the offsets between two intersections. Such pre-calculated offsets are not suitable for dynamic traffic environments. To incorporate cooperation in reinforcement learning (RL), two typical approaches are proposed to take the influence of other agents into consideration: (1) learning the communications (i.e., the representation of influences between agents) and (2) learning joint actions for agents. While joint modeling of actions has shown a preferred trend in recent studies, an in-depth study of improving the learning of communications between agents has not been systematically studied in the context of traffic signal control. To learn the communications between agents, in this paper, we propose to use graph attentional network to facilitate cooperation. Specifically, for a target intersection in a network, our proposed model, CoLight, cannot only incorporate the influences of neighboring intersections but learn to differentiate their impacts to the target intersection. To the best of our knowledge, we are the first to use graph attentional network in the setting of reinforcement learning for traffic signal control. In experiments, we demonstrate that by learning the communication, the proposed model can achieve surprisingly good performance, whereas the existing approaches based on joint action modeling fail to learn well.


A Novel Multi-Agent Deep RL Approach for Traffic Signal Control

As travel demand increases and urban traffic condition becomes more comp...

Multi-agent Reinforcement Learning for Regional Signal control in Large-scale Grid Traffic network

Adaptive traffic signal control with Multi-agent Reinforcement Learning(...

Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning

Inefficient traffic control may cause numerous problems such as traffic ...

Effects of Smart Traffic Signal Control on Air Quality

Adaptive traffic signal control (ATSC) in urban traffic networks poses a...

Adjust Planning Strategies to Accommodate Reinforcement Learning Agents

In agent control issues, the idea of combining reinforcement learning an...

Quantifying the Impact of Non-Stationarity in Reinforcement Learning-Based Traffic Signal Control

In reinforcement learning (RL), dealing with non-stationarity is a chall...

Multi-Agent Reinforcement Learning for Traffic Signal Control through Universal Communication Method

How to coordinate the communication among intersections effectively in r...

Please sign up or login with your details

Forgot password? Click here to reset