Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark

10/12/2015
by   Alexander Lavin, et al.
0

Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations; examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment of open-source tools to test and measure anomaly detection algorithms on streaming data. The perfect detector would detect all anomalies as soon as possible, trigger no false alarms, work with real-world time-series data across a variety of domains, and automatically adapt to changing statistics. Rewarding these characteristics is formalized in NAB, using a scoring algorithm designed for streaming data. NAB evaluates detectors on a benchmark dataset with labeled, real-world time-series data. We present these components, and give results and analyses for several open source, commercially-used algorithms. The goal for NAB is to provide a standard, open source framework with which the research community can compare and evaluate different algorithms for detecting anomalies in streaming data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/08/2016

Real-Time Anomaly Detection for Streaming Analytics

Much of the worlds data is streaming, time-series data, where anomalies ...
research
07/30/2020

Anomaly Detection at Scale: The Case for Deep Distributional Time Series Models

This paper introduces a new methodology for detecting anomalies in time ...
research
09/13/2022

A Meta-level Analysis of Online Anomaly Detectors

Real-time detection of anomalies in streaming data is receiving increasi...
research
09/16/2019

No Free Lunch But A Cheaper Supper: A General Framework for Streaming Anomaly Detection

Over the past years, there has been an increased research interest in th...
research
11/13/2019

Real-Time Anomaly Detection for Advanced Manufacturing: Improving on Twitter's State of the Art

The detection of anomalies in real time is paramount to maintain perform...
research
10/16/2014

Multi-Level Anomaly Detection on Time-Varying Graph Data

This work presents a novel modeling and analysis framework for graph seq...
research
11/14/2019

Real-time Anomaly Detection and Classification in Streaming PMU Data

Ensuring secure and reliable operations of the power grid is a primary c...

Please sign up or login with your details

Forgot password? Click here to reset