Advanced Scaling Methods for VNF deployment with Reinforcement Learning

by   Namjin Seo, et al.

Network function virtualization (NFV) and software-defined network (SDN) have become emerging network paradigms, allowing virtualized network function (VNF) deployment at a low cost. Even though VNF deployment can be flexible, it is still challenging to optimize VNF deployment due to its high complexity. Several studies have approached the task as dynamic programming, e.g., integer linear programming (ILP). However, optimizing VNF deployment for highly complex networks remains a challenge. Alternatively, reinforcement learning (RL) based approaches have been proposed to optimize this task, especially to employ a scaling action-based method which can deploy VNFs within less computational time. However, the model architecture can be improved further to generalize to the different networking settings. In this paper, we propose an enhanced model which can be adapted to more general network settings. We adopt the improved GNN architecture and a few techniques to obtain a better node representation for the VNF deployment task. Furthermore, we apply a recently proposed RL method, phasic policy gradient (PPG), to leverage the shared representation of the service function chain (SFC) generation model from the value function. We evaluate the proposed method in various scenarios, achieving a better QoS with minimum resource utilization compared to the previous methods. Finally, as a qualitative evaluation, we analyze our proposed encoder's representation for the nodes, which shows a more disentangled representation.


page 1

page 2

page 3

page 4


Policy Gradient for Reinforcement Learning with General Utilities

In Reinforcement Learning (RL), the goal of agents is to discover an opt...

Maximum Entropy Dueling Network Architecture

In recent years, there have been many deep structures for Reinforcement ...

A Queuing based Dynamic Auto Scaling Algorithm for the LTE EPC Control Plane

Network Slicing (NS) is expected to be a key functionality of the upcomi...

Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality

Deployment efficiency is an important criterion for many real-world appl...

Hierarchical Reinforcement Learning Framework for Stochastic Spaceflight Campaign Design

This paper develops a hierarchical reinforcement learning architecture f...

Dynamic Virtual Network Embedding Algorithm based on Graph Convolution Neural Network and Reinforcement Learning

Network virtualization (NV) is a technology with broad application prosp...

Graph Neural Network Based Node Deployment for Throughput Enhancement

The recent rapid growth in mobile data traffic entails a pressing demand...

Please sign up or login with your details

Forgot password? Click here to reset