ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxation

by   Chuangchuang Sun, et al.

In a multirobot system, a number of cyber-physical attacks (e.g., communication hijack, observation perturbations) can challenge the robustness of agents. This robustness issue worsens in multiagent reinforcement learning because there exists the non-stationarity of the environment caused by simultaneously learning agents whose changing policies affect the transition and reward functions. In this paper, we propose a minimax MARL approach to infer the worst-case policy update of other agents. As the minimax formulation is computationally intractable to solve, we apply the convex relaxation of neural networks to solve the inner minimization problem. Such convex relaxation enables robustness in interacting with peer agents that may have significantly different behaviors and also achieves a certified bound of the original optimization problem. We evaluate our approach on multiple mixed cooperative-competitive tasks and show that our method outperforms the previous state of the art approaches on this topic.


page 1

page 2

page 3

page 4


Robust Deep Reinforcement Learning through Adversarial Loss

Deep neural networks, including reinforcement learning agents, have been...

A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning

A fundamental challenge in multiagent reinforcement learning is to learn...

Robustness Testing for Multi-Agent Reinforcement Learning: State Perturbations on Critical Agents

Multi-Agent Reinforcement Learning (MARL) has been widely applied in man...

Robust Reinforcement Learning through Efficient Adversarial Herding

Although reinforcement learning (RL) is considered the gold standard for...

DeepSplit: Scalable Verification of Deep Neural Networks via Operator Splitting

Analyzing the worst-case performance of deep neural networks against inp...

Influencing Long-Term Behavior in Multiagent Reinforcement Learning

The main challenge of multiagent reinforcement learning is the difficult...

Adversarial training for continuous robustness control problem in power systems

We propose a new adversarial training approach for injecting robustness ...

Please sign up or login with your details

Forgot password? Click here to reset