Graph Reinforcement Learning Application to Co-operative Decision-Making in Mixed Autonomy Traffic: Framework, Survey, and Challenges

by   Qi Liu, et al.

Proper functioning of connected and automated vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaboration decision-making for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) has been widely used in solving decision-making problems. However, the existing DRL-based methods have been mainly focused on solving the decision-making of a single CAV. Using the existing DRL-based methods in mixed autonomy traffic cannot accurately represent the mutual effects of vehicles and model dynamic traffic environments. To address these shortcomings, this article proposes a graph reinforcement learning (GRL) approach for multi-agent decision-making of CAVs in mixed autonomy traffic. First, a generic and modular GRL framework is designed. Then, a systematic review of DRL and GRL methods is presented, focusing on the problems addressed in recent research. Moreover, a comparative study on different GRL methods is further proposed based on the designed framework to verify the effectiveness of GRL methods. Results show that the GRL methods can well optimize the performance of multi-agent decision-making for CAVs in mixed autonomy traffic compared to the DRL methods. Finally, challenges and future research directions are summarized. This study can provide a valuable research reference for solving the multi-agent decision-making problems of CAVs in mixed autonomy traffic and can promote the implementation of GRL-based methods into intelligent transportation systems. The source code of our work can be found at


page 1

page 10

page 13

page 17


Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments

An efficient and reliable multi-agent decision-making system is highly d...

Towards Efficient Connected and Automated Driving System via Multi-agent Graph Reinforcement Learning

Connected and automated vehicles (CAVs) have attracted more and more att...

Transfer Learning versus Multi-agent Learning regarding Distributed Decision-Making in Highway Traffic

Transportation and traffic are currently undergoing a rapid increase in ...

A Review of Situation Awareness Assessment Approaches in Aviation Environments

Situation awareness (SA) is an important constituent in human informatio...

Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey

Recent technology development brings the booming of numerous new Demand-...

Generative AI-enabled Vehicular Networks: Fundamentals, Framework, and Case Study

Recognizing the tremendous improvements that the integration of generati...

Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospects

Significant advances have recently been achieved in Multi-Agent Reinforc...

Please sign up or login with your details

Forgot password? Click here to reset