Sequential Multivariate Change Detection with Calibrated and Memoryless False Detection Rates

by   Oliver Cobb, et al.

Responding appropriately to the detections of a sequential change detector requires knowledge of the rate at which false positives occur in the absence of change. When the pre-change and post-change distributions are unknown, setting detection thresholds to achieve a desired false positive rate is challenging, even when there exists a large number of samples from the reference distribution. Existing works resort to setting time-invariant thresholds that focus on the expected runtime of the detector in the absence of change, either bounding it loosely from below or targeting it directly but with asymptotic arguments that we show cause significant miscalibration in practice. We present a simulation-based approach to setting time-varying thresholds that allows a desired expected runtime to be targeted with a 20x reduction in miscalibration whilst additionally keeping the false positive rate constant across time steps. Whilst the approach to threshold setting is metric agnostic, we show that when using the popular and powerful quadratic time MMD estimator, thoughtful structuring of the computation can reduce the cost during configuration from O(N^2B) to O(N^2+NB) and during operation from O(N^2) to O(N), where N is the number of reference samples and B the number of bootstrap samples. Code is made available as part of the open-source Python library .


On agnostic post hoc approaches to false positive control

This document is a book chapter which gives a partial survey on post hoc...

Reducing sequential change detection to sequential estimation

We consider the problem of sequential change detection, where the goal i...

Detecting Deforestation from Sentinel-1 Data in the Absence of Reliable Reference Data

Forests are vital for the wellbeing of our planet. Large and small scale...

Separating an Outlier from a Change

We study the quickest change detection problem with an unknown post-chan...

Optimizing Lead Time in Fall Detection for a Planar Bipedal Robot

For legged robots to operate in complex terrains, they must be robust to...

The False Positive Control Lasso

In high dimensional settings where a small number of regressors are expe...

Quickest Changepoint Detection in General Multistream Stochastic Models: Recent Results, Applications and Future Challenges

Modern information systems generate large volumes of data with anomalies...

Please sign up or login with your details

Forgot password? Click here to reset