Bayesian Anomaly Detection and Classification

02/22/2019
by   Ethan Roberts, et al.
AIMS South Africa
myUCT
0

Statistical uncertainties are rarely incorporated in machine learning algorithms, especially for anomaly detection. Here we present the Bayesian Anomaly Detection And Classification (BADAC) formalism, which provides a unified statistical approach to classification and anomaly detection within a hierarchical Bayesian framework. BADAC deals with uncertainties by marginalising over the unknown, true, value of the data. Using simulated data with Gaussian noise, BADAC is shown to be superior to standard algorithms in both classification and anomaly detection performance in the presence of uncertainties, though with significantly increased computational cost. Additionally, BADAC provides well-calibrated classification probabilities, valuable for use in scientific pipelines. We show that BADAC can work in online mode and is fairly robust to model errors, which can be diagnosed through model-selection methods. In addition it can perform unsupervised new class detection and can naturally be extended to search for anomalous subsets of data. BADAC is therefore ideal where computational cost is not a limiting factor and statistical rigour is important. We discuss approximations to speed up BADAC, such as the use of Gaussian processes, and finally introduce a new metric, the Rank-Weighted Score (RWS), that is particularly suited to evaluating the ability of algorithms to detect anomalies.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/19/2020

anomaly : Detection of Anomalous Structure in Time Series Data

One of the contemporary challenges in anomaly detection is the ability t...
11/11/2016

Low Latency Anomaly Detection and Bayesian Network Prediction of Anomaly Likelihood

We develop a supervised machine learning model that detects anomalies in...
12/19/2019

Normalizing flows for deep anomaly detection

Anomaly detection for complex data is a challenging task from the perspe...
09/04/2020

Simulation-Assisted Decorrelation for Resonant Anomaly Detection

A growing number of weak- and unsupervised machine learning approaches t...
10/30/2018

Quickest Detection Of Deviations From Periodic Statistical Behavior

A new class of stochastic processes called independent and periodically ...

Please sign up or login with your details

Forgot password? Click here to reset