A Critical Evaluation of Open-World Machine Learning

by   Liwei Song, et al.

Open-world machine learning (ML) combines closed-world models trained on in-distribution data with out-of-distribution (OOD) detectors, which aim to detect and reject OOD inputs. Previous works on open-world ML systems usually fail to test their reliability under diverse, and possibly adversarial conditions. Therefore, in this paper, we seek to understand how resilient are state-of-the-art open-world ML systems to changes in system components? With our evaluation across 6 OOD detectors, we find that the choice of in-distribution data, model architecture and OOD data have a strong impact on OOD detection performance, inducing false positive rates in excess of 70%. We further show that OOD inputs with 22 unintentional corruptions or adversarial perturbations render open-world ML systems unusable with false positive rates of up to 100%. To increase the resilience of open-world ML, we combine robust classifiers with OOD detection techniques and uncover a new trade-off between OOD detection and robustness.


Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples

A large body of recent work has investigated the phenomenon of evasion a...

Towards resilient machine learning for ransomware detection

There has been a surge of interest in using machine learning (ML) to aut...

Automatic Open-World Reliability Assessment

Image classification in the open-world must handle out-of-distribution (...

Towards a Resilient Machine Learning Classifier – a Case Study of Ransomware Detection

The damage caused by crypto-ransomware, due to encryption, is difficult ...

The More, the Better? A Study on Collaborative Machine Learning for DGA Detection

Domain generation algorithms (DGAs) prevent the connection between a bot...

The games we play: critical complexity improves machine learning

When mathematical modelling is applied to capture a complex system, mult...

SQLi Detection with ML: A data-source perspective

Almost 50 years after the invention of SQL, injection attacks are still ...

Please sign up or login with your details

Forgot password? Click here to reset