Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model

by   Tianyu Shi, et al.

In the coming years and decades, autonomous vehicles (AVs) will become increasingly prevalent, offering new opportunities for safer and more convenient travel and potentially smarter traffic control methods exploiting automation and connectivity. Car following is a prime function in autonomous driving. Car following based on reinforcement learning has received attention in recent years with the goal of learning and achieving performance levels comparable to humans. However, most existing RL methods model car following as a unilateral problem, sensing only the vehicle ahead. Recent literature, however, Wang and Horn [16] has shown that bilateral car following that considers the vehicle ahead and the vehicle behind exhibits better system stability. In this paper we hypothesize that this bilateral car following can be learned using RL, while learning other goals such as efficiency maximisation, jerk minimization, and safety rewards leading to a learned model that outperforms human driving. We propose and introduce a Deep Reinforcement Learning (DRL) framework for car following control by integrating bilateral information into both state and reward function based on the bilateral control model (BCM) for car following control. Furthermore, we use a decentralized multi-agent reinforcement learning framework to generate the corresponding control action for each agent. Our simulation results demonstrate that our learned policy is better than the human driving policy in terms of (a) inter-vehicle headways, (b) average speed, (c) jerk, (d) Time to Collision (TTC) and (e) string stability.


page 1

page 6


Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving

A model used for velocity control during car following was proposed base...

Longitudinal Dynamic versus Kinematic Models for Car-following Control Using Deep Reinforcement Learning

The majority of current studies on autonomous vehicle control via deep r...

Image-Based Conditioning for Action Policy Smoothness in Autonomous Miniature Car Racing with Reinforcement Learning

In recent years, deep reinforcement learning has achieved significant re...

Towards Efficient Connected and Automated Driving System via Multi-agent Graph Reinforcement Learning

Connected and automated vehicles (CAVs) have attracted more and more att...

Interactive Car-Following: Matters but NOT Always

Following a leading vehicle is a daily but challenging task because it r...

Multi-vehicle Platoon Overtaking Using NoisyNet Multi-Agent Deep Q-Learning Network

With the recent advancements in Vehicle-to-Vehicle communication technol...

Image-based Regularization for Action Smoothness in Autonomous Miniature Racing Car with Deep Reinforcement Learning

Deep reinforcement learning has achieved significant results in low-leve...

Please sign up or login with your details

Forgot password? Click here to reset