Perimeter Control Using Deep Reinforcement Learning: A Model-free Approach towards Homogeneous Flow Rate Optimization

by   Xiaocan Li, et al.

Perimeter control maintains high traffic efficiency within protected regions by controlling transfer flows among regions to ensure that their traffic densities are below critical values. Existing approaches can be categorized as either model-based or model-free, depending on whether they rely on network transmission models (NTMs) and macroscopic fundamental diagrams (MFDs). Although model-based approaches are more data efficient and have performance guarantees, they are inherently prone to model bias and inaccuracy. For example, NTMs often become imprecise for a large number of protected regions, and MFDs can exhibit scatter and hysteresis that are not captured in existing model-based works. Moreover, no existing studies have employed reinforcement learning for homogeneous flow rate optimization in microscopic simulation, where spatial characteristics, vehicle-level information, and metering realizations – often overlooked in macroscopic simulations – are taken into account. To circumvent issues of model-based approaches and macroscopic simulation, we propose a model-free deep reinforcement learning approach that optimizes the flow rate homogeneously at the perimeter at the microscopic level. Results demonstrate that our model-free reinforcement learning approach without any knowledge of NTMs or MFDs can compete and match the performance of a model-based approach, and exhibits enhanced generalizability and scalability.


Curious Meta-Controller: Adaptive Alternation between Model-Based and Model-Free Control in Deep Reinforcement Learning

Recent success in deep reinforcement learning for continuous control has...

Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning

Model-free deep reinforcement learning algorithms have been shown to be ...

Do recent advancements in model-based deep reinforcement learning really improve data efficiency?

Reinforcement learning (RL) has seen great advancements in the past few ...

Behaviorally Grounded Model-Based and Model Free Cost Reduction in a Simulated Multi-Echelon Supply Chain

Amplification and phase shift in ordering signals, commonly referred to ...

Adaptive learning for financial markets mixing model-based and model-free RL for volatility targeting

Model-Free Reinforcement Learning has achieved meaningful results in sta...

Deep Reinforcement Learning for Concentric Tube Robot Path Planning

As surgical interventions trend towards minimally invasive approaches, C...

Probabilistic Programming Bots in Intuitive Physics Game Play

Recent findings suggest that humans deploy cognitive mechanism of physic...

Please sign up or login with your details

Forgot password? Click here to reset