Robust and Scalable Routing with Multi-Agent Deep Reinforcement Learning for MANETs

by   Saeed Kaviani, et al.

Highly dynamic mobile ad-hoc networks (MANETs) are continuing to serve as one of the most challenging environments to develop and deploy robust, efficient, and scalable routing protocols. In this paper, we present DeepCQ+ routing which, in a novel manner, integrates emerging multi-agent deep reinforcement learning (MADRL) techniques into existing Q-learning-based routing protocols and their variants, and achieves persistently higher performance across a wide range of MANET configurations while training only on a limited range of network parameters and conditions. Quantitatively, DeepCQ+ shows consistently higher end-to-end throughput with lower overhead compared to its Q-learning-based counterparts with the overall gain of 10-15 and more significantly, DeepCQ+ maintains remarkably similar performance gains under many scenarios that it was not trained for in terms of network sizes, mobility conditions, and traffic dynamics. To the best of our knowledge, this is the first successful demonstration of MADRL for the MANET routing problem that achieves and maintains a high degree of scalability and robustness even in the environments that are outside the trained range of scenarios. This implies that the proposed hybrid design approach of DeepCQ+ that combines MADRL and Q-learning significantly increases its practicality and explainability because the real-world MANET environment will likely vary outside the trained range of MANET scenarios.


page 1

page 9

page 10


DeepCQ+: Robust and Scalable Routing with Multi-Agent Deep Reinforcement Learning for Highly Dynamic Networks

Highly dynamic mobile ad-hoc networks (MANETs) remain as one of the most...

DeepMPR: Enhancing Opportunistic Routing in Wireless Networks through Multi-Agent Deep Reinforcement Learning

Opportunistic routing relies on the broadcast capability of wireless net...

Scalable Deep Reinforcement Learning for Routing and Spectrum Access in Physical Layer

This paper proposes a novel and scalable reinforcement learning approach...

DeepADMR: A Deep Learning based Anomaly Detection for MANET Routing

We developed DeepADMR, a novel neural anomaly detector for the deep rein...

A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization

In this paper we design and evaluate a Deep-Reinforcement Learning agent...

Towards Machine Learning-Enabled Context Adaption for Reliable Aerial Mesh Routing

In this paper, we present Context-Adaptive PARRoT (CA-PARRoT) as an exte...

Reinforcing Reachable Routes

This paper studies the evaluation of routing algorithms from the perspec...

Please sign up or login with your details

Forgot password? Click here to reset