TSB-UAD: An End-to-End Benchmark Suite for Univariate Time-Series Anomaly Detection

Download PDF

Abstract

The detection of anomalies in time-series data has gained ample academic and industrial attention. Unfortunately, there is currently no consensus on using a single benchmark for assessing the performance of time-series anomaly detection methods. It has become common practice to use (i) proprietary or synthetic data, often biased to support particular claims; or (ii) a limited collection of publicly available datasets. Consequently, we often observe methods performing exceptionally well in one dataset but surprisingly poorly in another, creating an illusion of progress. To address the aforementioned issues, we thoroughly studied over one hundred papers to identify, collect, and systematically process and format relevant datasets proposed in the literature in the past decades. We summarize our effort in TSB-UAD, a new benchmark to ease the evaluation of time-series anomaly detection methods. Overall, TSB-UAD contains $12686$ time series with labeled anomalies spanning different domains with high variability of anomaly types, ratios, and sizes. Specifically, TSB-UAD includes $10$ previously proposed datasets containing $900$ time series and we contribute two collections of datasets. Specifically, we generate $958$ time series using a principled methodology for transforming $126$ time-series classification datasets into time series with labeled anomalies. In addition, we present a set of data transformations with which we introduce new anomalies in the public datasets, resulting in $10828$ time series with varying complexity for anomaly detection. Finally, we present an initial evaluation of representative methods demonstrating that TSB-UAD is a robust resource for anomaly detection evaluation. We make our data and code available at \url{www.timeseries.org/TSB-UAD}. We believe TSB-UAD can provide a valuable, reproducible, and frequently updated resource to the community to establish a leaderboard of state-of-the-art time-series anomaly detection methods.