End-to-end Robustness for Sensing-Reasoning Machine Learning Pipelines

by   Zhuolin Yang, et al.

As machine learning (ML) being applied to many mission-critical scenarios, certifying ML model robustness becomes increasingly important. Many previous works focuses on the robustness of independent ML and ensemble models, and can only certify a very small magnitude of the adversarial perturbation. In this paper, we take a different viewpoint and improve learning robustness by going beyond independent ML and ensemble models. We aim at promoting the generic Sensing-Reasoning machine learning pipeline which contains both the sensing (e.g. deep neural networks) and reasoning (e.g. Markov logic networks (MLN)) components enriched with domain knowledge. Can domain knowledge help improve learning robustness? Can we formally certify the end-to-end robustness of such an ML pipeline? We first theoretically analyze the computational complexity of checking the provable robustness in the reasoning component. We then derive the provable robustness bound for several concrete reasoning components. We show that for reasoning components such as MLN and a specific family of Bayesian networks it is possible to certify the robustness of the whole pipeline even with a large magnitude of perturbation which cannot be certified by existing work. Finally, we conduct extensive real-world experiments on large scale datasets to evaluate the certified robustness for Sensing-Reasoning ML pipelines.


Testing the Robustness of AutoML Systems

Automated machine learning (AutoML) systems aim at finding the best mach...

CARE: Certifiably Robust Learning with Reasoning via Variational Inference

Despite great recent advances achieved by deep neural networks (DNNs), t...

Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach

Classical Machine Learning (ML) pipelines often comprise of multiple ML ...

Towards Observability for Machine Learning Pipelines

Software organizations are increasingly incorporating machine learning (...

Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attacks

Despite the great successes achieved by deep neural networks (DNNs), rec...

End-to-End Rationale Reconstruction

The logic behind design decisions, called design rationale, is very valu...

Please sign up or login with your details

Forgot password? Click here to reset