STDPG: A Spatio-Temporal Deterministic Policy Gradient Agent for Dynamic Routing in SDN

by   Juan Chen, et al.

Dynamic routing in software-defined networking (SDN) can be viewed as a centralized decision-making problem. Most of the existing deep reinforcement learning (DRL) agents can address it, thanks to the deep neural network (DNN)incorporated. However, fully-connected feed-forward neural network (FFNN) is usually adopted, where spatial correlation and temporal variation of traffic flows are ignored. This drawback usually leads to significantly high computational complexity due to large number of training parameters. To overcome this problem, we propose a novel model-free framework for dynamic routing in SDN, which is referred to as spatio-temporal deterministic policy gradient (STDPG) agent. Both the actor and critic networks are based on identical DNN structure, where a combination of convolutional neural network (CNN) and long short-term memory network (LSTM) with temporal attention mechanism, CNN-LSTM-TAM, is devised. By efficiently exploiting spatial and temporal features, CNNLSTM-TAM helps the STDPG agent learn better from the experience transitions. Furthermore, we employ the prioritized experience replay (PER) method to accelerate the convergence of model training. The experimental results show that STDPG can automatically adapt for current network environment and achieve robust convergence. Compared with a number state-ofthe-art DRL agents, STDPG achieves better routing solutions in terms of the average end-to-end delay.


Experience-driven Networking: A Deep Reinforcement Learning based Approach

Modern communication networks have become very complicated and highly dy...

DRL-M4MR: An Intelligent Multicast Routing Approach Based on DQN Deep Reinforcement Learning in SDN

Traditional multicast routing methods have some problems in constructing...

A Deep Spatio-Temporal Fuzzy Neural Network for Passenger Demand Prediction

In spite of its importance, passenger demand prediction is a highly chal...

Don't Watch Me: A Spatio-Temporal Trojan Attack on Deep-Reinforcement-Learning-Augment Autonomous Driving

Deep reinforcement learning (DRL) is one of the most popular algorithms ...

HSTF-Model: an HTTP-based Trojan Detection Model via the Hierarchical Spatio-Temporal Features of Traffics

HTTP-based Trojan is extremely threatening, and it is difficult to be ef...

Distributed Uplink Beamforming in Cell-Free Networks Using Deep Reinforcement Learning

The emergence of new wireless technologies together with the requirement...

HyMER: A Hybrid Machine Learning Framework for Energy Efficient Routing in SDN

Combining the capabilities of the programmability of networks by SDN and...

Please sign up or login with your details

Forgot password? Click here to reset