Little Help Makes a Big Difference: Leveraging Active Learning to Improve Unsupervised Time Series Anomaly Detection

by   Hamza Bodor, et al.
HUAWEI Technologies Co., Ltd.

Key Performance Indicators (KPI), which are essentially time series data, have been widely used to indicate the performance of telecom networks. Based on the given KPIs, a large set of anomaly detection algorithms have been deployed for detecting the unexpected network incidents. Generally, unsupervised anomaly detection algorithms gain more popularity than the supervised ones, due to the fact that labeling KPIs is extremely time- and resource-consuming, and error-prone. However, those unsupervised anomaly detection algorithms often suffer from excessive false alarms, especially in the presence of concept drifts resulting from network re-configurations or maintenance. To tackle this challenge and improve the overall performance of unsupervised anomaly detection algorithms, we propose to use active learning to introduce and benefit from the feedback of operators, who can verify the alarms (both false and true ones) and label the corresponding KPIs with reasonable effort. Specifically, we develop three query strategies to select the most informative and representative samples to label. We also develop an efficient method to update the weights of Isolation Forest and optimally adjust the decision threshold, so as to eventually improve the performance of detection model. The experiments with one public dataset and one proprietary dataset demonstrate that our active learning empowered anomaly detection pipeline could achieve performance gain, in terms of F1-score, more than 50 existing active learning based methods by approximately 6 significantly reduced budget (the ratio of samples to be labeled).


page 1

page 2

page 3

page 4


RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learning

We introduce a new semi-supervised, time series anomaly detection algori...

Label-Efficient Interactive Time-Series Anomaly Detection

Time-series anomaly detection is an important task and has been widely a...

Active Learning-based Isolation Forest (ALIF): Enhancing Anomaly Detection in Decision Support Systems

The detection of anomalous behaviours is an emerging need in many applic...

Active Anomaly Detection via Ensembles

In critical applications of anomaly detection including computer securit...

How to Allocate your Label Budget? Choosing between Active Learning and Learning to Reject in Anomaly Detection

Anomaly detection attempts at finding examples that deviate from the exp...

Active Learning for Network Intrusion Detection

Network operators are generally aware of common attack vectors that they...

Please sign up or login with your details

Forgot password? Click here to reset