Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series

by   Sondre Sørbø, et al.

The field of time series anomaly detection is constantly advancing, with several methods available, making it a challenge to determine the most appropriate method for a specific domain. The evaluation of these methods is facilitated by the use of metrics, which vary widely in their properties. Despite the existence of new evaluation metrics, there is limited agreement on which metrics are best suited for specific scenarios and domain, and the most commonly used metrics have faced criticism in the literature. This paper provides a comprehensive overview of the metrics used for the evaluation of time series anomaly detection methods, and also defines a taxonomy of these based on how they are calculated. By defining a set of properties for evaluation metrics and a set of specific case studies and experiments, twenty metrics are analyzed and discussed in detail, highlighting the unique suitability of each for specific tasks. Through extensive experimentation and analysis, this paper argues that the choice of evaluation metric must be made with care, taking into account the specific requirements of the task at hand.


page 1

page 2

page 3

page 4


A Comparative Study on Unsupervised Anomaly Detection for Time Series: Experiments and Analysis

The continued digitization of societal processes translates into a proli...

SoftED: Metrics for Soft Evaluation of Time Series Event Detection

Time series event detection methods are evaluated mainly by standard cla...

An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series

Several techniques for multivariate time series anomaly detection have b...

Local Evaluation of Time Series Anomaly Detection Algorithms

In recent years, specific evaluation metrics for time series anomaly det...

Anomaly Detection in Predictive Maintenance: A New Evaluation Framework for Temporal Unsupervised Anomaly Detection Algorithms

The research in anomaly detection lacks a unified definition of what rep...

tegdet: An extensible Python Library for Anomaly Detection using Time-Evolving Graphs

This paper presents a new Python library for anomaly detection in unsupe...

Streamlined Framework for Agile Forecasting Model Development towards Efficient Inventory Management

This paper proposes a framework for developing forecasting models by str...

Please sign up or login with your details

Forgot password? Click here to reset