CARE: Certifiably Robust Learning with Reasoning via Variational Inference

09/12/2022
by   Jiawei Zhang, et al.
2

Despite great recent advances achieved by deep neural networks (DNNs), they are often vulnerable to adversarial attacks. Intensive research efforts have been made to improve the robustness of DNNs; however, most empirical defenses can be adaptively attacked again, and the theoretically certified robustness is limited, especially on large-scale datasets. One potential root cause of such vulnerabilities for DNNs is that although they have demonstrated powerful expressiveness, they lack the reasoning ability to make robust and reliable predictions. In this paper, we aim to integrate domain knowledge to enable robust learning with the reasoning paradigm. In particular, we propose a certifiably robust learning with reasoning pipeline (CARE), which consists of a learning component and a reasoning component. Concretely, we use a set of standard DNNs to serve as the learning component to make semantic predictions, and we leverage the probabilistic graphical models, such as Markov logic networks (MLN), to serve as the reasoning component to enable knowledge/logic reasoning. However, it is known that the exact inference of MLN (reasoning) is #P-complete, which limits the scalability of the pipeline. To this end, we propose to approximate the MLN inference via variational inference based on an efficient expectation maximization algorithm. In particular, we leverage graph convolutional networks (GCNs) to encode the posterior distribution during variational inference and update the parameters of GCNs (E-step) and the weights of knowledge rules in MLN (M-step) iteratively. We conduct extensive experiments on different datasets and show that CARE achieves significantly higher certified robustness compared with the state-of-the-art baselines. We additionally conducted different ablation studies to demonstrate the empirical robustness of CARE and the effectiveness of different knowledge integration.

READ FULL TEXT

page 1

page 3

page 7

page 10

page 12

page 18

page 19

page 20

research
06/11/2021

Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attacks

Despite the great successes achieved by deep neural networks (DNNs), rec...
research
01/29/2020

Efficient Probabilistic Logic Reasoning with Graph Neural Networks

Markov Logic Networks (MLNs), which elegantly combine logic rules and pr...
research
02/28/2020

End-to-end Robustness for Sensing-Reasoning Machine Learning Pipelines

As machine learning (ML) being applied to many mission-critical scenario...
research
09/09/2020

SoK: Certified Robustness for Deep Neural Networks

Great advancement in deep neural networks (DNNs) has led to state-of-the...
research
08/25/2019

Tutorial and Survey on Probabilistic Graphical Model and Variational Inference in Deep Reinforcement Learning

Probabilistic Graphical Modeling and Variational Inference play an impor...
research
11/10/2021

Robust Learning via Ensemble Density Propagation in Deep Neural Networks

Learning in uncertain, noisy, or adversarial environments is a challengi...

Please sign up or login with your details

Forgot password? Click here to reset