Deep Reinforcement Learning for IRS Phase Shift Design in Spatiotemporally Correlated Environments

by   Spilios Evmorfos, et al.

The paper studies the problem of designing the Intelligent Reflecting Surface (IRS) phase shifters for Multiple Input Single Output (MISO) communication systems in spatiotemporally correlated channel environments, where the destination can move within a confined area. The objective is to maximize the expected sum of SNRs at the receiver over infinite time horizons. The problem formulation gives rise to a Markov Decision Process (MDP). We propose a deep actor-critic algorithm that accounts for channel correlations and destination motion by constructing the state representation to include the current position of the receiver and the phase shift values and receiver positions that correspond to a window of previous time steps. The channel variability induces high frequency components on the spectrum of the underlying value function. We propose the preprocessing of the critic's input with a Fourier kernel which enables stable value learning. Finally, we investigate the use of the destination SNR as a component of the designed MDP state, which is common practice in previous work. We provide empirical evidence that, when the channels are spatiotemporally correlated, the inclusion of the SNR in the state representation interacts with function approximation in ways that inhibit convergence.


Deep Reinforcement Learning for Intelligent Reflecting Surface-assisted D2D Communications

In this paper, we propose a deep reinforcement learning (DRL) approach f...

On the Robustness of Deep Reinforcement Learning in IRS-Aided Wireless Communications Systems

We consider an Intelligent Reflecting Surface (IRS)-aided multiple-input...

Managing engineering systems with large state and action spaces through deep reinforcement learning

Decision-making for engineering systems can be efficiently formulated as...

Deep Reinforcement Learning Based Intelligent Reflecting Surface Optimization for TDD MultiUser MIMO Systems

In this letter, we investigate the discrete phase shift design of the in...

Dual-Function Radar-Communication System Aided by Intelligent Reflecting Surfaces

We propose a novel design of a dual-function radar communication (DFRC) ...

Quantized Phase-Shift Design of Active IRS for Integrated Sensing and Communications

Integrated sensing and communications (ISAC) is a spectrum-sharing parad...

Please sign up or login with your details

Forgot password? Click here to reset