Load Balancing for Ultra-Dense Networks: A Deep Reinforcement Learning Based Approach

by   Yue Xu, et al.

In this paper, we propose a deep reinforcement learning (DRL) based mobility load balancing (MLB) algorithm along with a two-layer architecture to solve the large-scale load balancing problem for ultra-dense networks (UDNs). Our contribution is three-fold. First, this work proposes a two-layer architecture to solve the large-scale load balancing problem in a self-organized manner. The proposed architecture can alleviate the global traffic variations by dynamically grouping small cells into self-organized clusters according to their historical loads, and further adapt to local traffic variations through intra-cluster load balancing afterwards. Second, for the intra-cluster load balancing, this paper proposes an off-policy DRL-based MLB algorithm to autonomously learn the optimal MLB policy under an asynchronous parallel learning framework, without any prior knowledge assumed over the underlying UDN environments. Moreover, the algorithm enables joint exploration with multiple behavior policies, such that the traditional MLB methods can be used to guide the learning process thereby improving the learning efficiency and stability. Third, this work proposes an offline-evaluation based safeguard mechanism to ensure that the online system can always operate with the optimal and well-trained MLB policy, which not only stabilizes the online performance but also enables the exploration beyond current policies to make full use of machine learning in a safe way. Empirical results verify that the proposed framework outperforms the existing MLB methods in general UDN environments featured with irregular network topologies, coupled interferences, and random user movements, in terms of the load balancing performance.


page 2

page 4

page 5

page 6

page 7

page 9

page 10

page 11


On the Robustness of Controlled Deep Reinforcement Learning for Slice Placement

The evaluation of the impact of using Machine Learning in the management...

Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center

This paper presents the network load balancing problem, a challenging re...

Deep Reinforcement Learning for Optimal Power Flow with Renewables Using Spatial-Temporal Graph Information

Renewable energy resources (RERs) have been increasingly integrated into...

Ad-load Balancing via Off-policy Learning in a Content Marketplace

Ad-load balancing is a critical challenge in online advertising systems,...

Stability of Traffic Load Balancing on Wireless Complex Networks

Load balancing between adjacent base stations (BS) is important for homo...

Handover Control in Wireless Systems via Asynchronous Multi-User Deep Reinforcement Learning

In this paper, we propose a two-layer framework to learn the optimal han...

Context-aware Group Buying in Ultra-dense Small Cell Networks: Unity is Strength

The ultra-dense small cell networks (SCNs) have been regarded as a promi...

Please sign up or login with your details

Forgot password? Click here to reset