Distributed Uplink Beamforming in Cell-Free Networks Using Deep Reinforcement Learning

by   Firas Fredj, et al.

The emergence of new wireless technologies together with the requirement of massive connectivity results in several technical issues such as excessive interference, high computational demand for signal processing, and lengthy processing delays. In this work, we propose several beamforming techniques for an uplink cell-free network with centralized, semi-distributed, and fully distributed processing, all based on deep reinforcement learning (DRL). First, we propose a fully centralized beamforming method that uses the deep deterministic policy gradient algorithm (DDPG) with continuous space. We then enhance this method by enabling distributed experience at access points (AP). Indeed, we develop a beamforming scheme that uses the distributed distributional deterministic policy gradients algorithm (D4PG) with the APs representing the distributed agents. Finally, to decrease the computational complexity, we propose a fully distributed beamforming scheme that divides the beamforming computations among APs. The results show that the D4PG scheme with distributed experience achieves the best performance irrespective of the network size. Furthermore, the proposed distributed beamforming technique performs better than the DDPG algorithm with centralized learning only for small-scale networks. The performance superiority of the DDPG model becomes more evident as the number of APs and/or users increases. Moreover, during the operation stage, all DRL models demonstrate a significantly shorter processing time than that of the conventional gradient descent (GD) solution.


page 20

page 22


Multiple Access in Dynamic Cell-Free Networks: Outage Performance and Deep Reinforcement Learning-Based Design

In future cell-free (or cell-less) wireless networks, a large number of ...

Deep Reinforcement Learning for Energy-Efficient Beamforming Design in Cell-Free Networks

Cell-free network is considered as a promising architecture for satisfyi...

Self-Organizing mmWave MIMO Cell-Free Networks With Hybrid Beamforming: A Hierarchical DRL-Based Design

In a cell-free wireless network, distributed access points (APs) jointly...

Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications

Intelligent reflecting surface (IRS) is a promising technology to assist...

Limited-Fronthaul Cell-Free Hybrid Beamforming with Distributed Deep Neural Network

Cell-free massive MIMO (CF-mMIMO) systems represent a promising approach...

Learning Deterministic Policy with Target for Power Control in Wireless Networks

Inter-Cell Interference Coordination (ICIC) is a promising way to improv...

STDPG: A Spatio-Temporal Deterministic Policy Gradient Agent for Dynamic Routing in SDN

Dynamic routing in software-defined networking (SDN) can be viewed as a ...

Please sign up or login with your details

Forgot password? Click here to reset