Deep Reinforcement Learning for Energy-Efficient Beamforming Design in Cell-Free Networks

by   Weilai Li, et al.

Cell-free network is considered as a promising architecture for satisfying more demands of future wireless networks, where distributed access points coordinate with an edge cloud processor to jointly provide service to a smaller number of user equipments in a compact area. In this paper, the problem of uplink beamforming design is investigated for maximizing the long-term energy efficiency (EE) with the aid of deep reinforcement learning (DRL) in the cell-free network. Firstly, based on the minimum mean square error channel estimation and exploiting successive interference cancellation for signal detection, the expression of signal to interference plus noise ratio (SINR) is derived. Secondly, according to the formulation of SINR, we define the long-term EE, which is a function of beamforming matrix. Thirdly, to address the dynamic beamforming design with continuous state and action space, a DRL-enabled beamforming design is proposed based on deep deterministic policy gradient (DDPG) algorithm by taking the advantage of its double-network architecture. Finally, the results of simulation indicate that the DDPG-based beamforming design is capable of converging to the optimal EE performance. Furthermore, the influence of hyper-parameters on the EE performance of the DDPG-based beamforming design is investigated, and it is demonstrated that an appropriate discount factor and hidden layers size can facilitate the EE performance.


page 1

page 4


Multiple Access in Dynamic Cell-Free Networks: Outage Performance and Deep Reinforcement Learning-Based Design

In future cell-free (or cell-less) wireless networks, a large number of ...

Distributed Uplink Beamforming in Cell-Free Networks Using Deep Reinforcement Learning

The emergence of new wireless technologies together with the requirement...

Self-Organizing mmWave MIMO Cell-Free Networks With Hybrid Beamforming: A Hierarchical DRL-Based Design

In a cell-free wireless network, distributed access points (APs) jointly...

Learning Deterministic Policy with Target for Power Control in Wireless Networks

Inter-Cell Interference Coordination (ICIC) is a promising way to improv...

Reinforcement Learning Assisted Beamforming for Inter-cell Interference Mitigation in 5G Massive MIMO Networks

Beamforming is an essential technology in the 5G massive multiple-input-...

Deep Reinforcement Learning Based Dynamic Power and Beamforming Design for Time-Varying Wireless Downlink Interference Channel

With the high development of wireless communication techniques, it is wi...

Cross-Layer Scheduling and Beamforming in Smart Grid Powered Small-Cell Networks

In the small-cell networks (SCNs) with multiple small-cell base stations...

Please sign up or login with your details

Forgot password? Click here to reset