Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in Multi-Agent RL

by   Xiangyu Liu, et al.

Most existing works consider direct perturbations of victim's state/action or the underlying transition dynamics to show vulnerability of reinforcement learning agents under adversarial attacks. However, such direct manipulation may not always be feasible in practice. In this paper, we consider another common and realistic attack setup: in a multi-agent RL setting with well-trained agents, during deployment time, the victim agent ν is exploited by an attacker who controls another agent α to act adversarially against the victim using an adversarial policy. Prior attack models under such setup do not consider that the attacker can confront resistance and thus can only take partial control of the agent α, as well as introducing perceivable “abnormal” behaviors that are easily detectable. A provable defense against these adversarial policies is also lacking. To resolve these issues, we introduce a more general attack formulation that models to what extent the adversary is able to control the agent to produce the adversarial policy. Based on such a generalized attack framework, the attacker can also regulate the state distribution shift caused by the attack through an attack budget, and thus produce stealthy adversarial policies that can exploit the victim agent. Furthermore, we provide the first provably robust defenses with convergence guarantee to the most robust victim policy via adversarial training with timescale separation, in sharp contrast to adversarial training in supervised learning which may only provide empirical defenses.


page 8

page 9


Sparse Adversarial Attack in Multi-agent Reinforcement Learning

Cooperative multi-agent reinforcement learning (cMARL) has many real app...

Training Automated Defense Strategies Using Graph-based Cyber Attack Simulations

We implemented and evaluated an automated cyber defense agent. The agent...

Two Can Play That Game: An Adversarial Evaluation of a Cyber-alert Inspection System

Cyber-security is an important societal concern. Cyber-attacks have incr...

Policy Smoothing for Provably Robust Reinforcement Learning

The study of provable adversarial robustness for deep neural network (DN...

Reducing Exploitability with Population Based Training

Self-play reinforcement learning has achieved state-of-the-art, and ofte...

Provable Defense against Backdoor Policies in Reinforcement Learning

We propose a provable defense mechanism against backdoor policies in rei...

Novel Stealthy Attack and Defense Strategies for Networked Control Systems

This paper studies novel attack and defense strategies, based on a class...

Please sign up or login with your details

Forgot password? Click here to reset