PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control

by   Dong Chen, et al.

This paper develops an efficient multi-agent deep reinforcement learning algorithm for cooperative controls in powergrids. Specifically, we consider the decentralized inverter-based secondary voltage control problem in distributed generators (DGs), which is first formulated as a cooperative multi-agent reinforcement learning (MARL) problem. We then propose a novel on-policy MARL algorithm, PowerNet, in which each agent (DG) learns a control policy based on (sub-)global reward but local states from its neighboring agents. Motivated by the fact that a local control from one agent has limited impact on agents distant from it, we exploit a novel spatial discount factor to reduce the effect from remote agents, to expedite the training process and improve scalability. Furthermore, a differentiable, learning-based communication protocol is employed to foster the collaborations among neighboring agents. In addition, to mitigate the effects of system uncertainty and random noise introduced during on-policy learning, we utilize an action smoothing factor to stabilize the policy execution. To facilitate training and evaluation, we develop PGSim, an efficient, high-fidelity powergrid simulation platform. Experimental results in two microgrid setups show that the developed PowerNet outperforms a conventional model-based control, as well as several state-of-the-art MARL algorithms. The decentralized learning scheme and high sample efficiency also make it viable to large-scale power grids.


page 1

page 4


Multi-agent Reinforcement Learning for Networked System Control

This paper considers multi-agent reinforcement learning (MARL) in networ...

Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet

In heterogeneous networks (HetNets), the overlap of small cells and the ...

Cooperative Heterogeneous Deep Reinforcement Learning

Numerous deep reinforcement learning agents have been proposed, and each...

Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning

Pursuit-evasion is the problem of capturing mobile targets with one or m...

Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning

This paper proposes a data-driven distributed voltage control approach b...

Learning to flock through reinforcement

Flocks of birds, schools of fish, insects swarms are examples of coordin...

Multi-Agent Reinforcement Learning with Common Policy for Antenna Tilt Optimization

This paper proposes a method for wireless network optimization applicabl...

Please sign up or login with your details

Forgot password? Click here to reset