Large-scale unusual time series detection

Rob J. Hyndman; Earo Wang; Nikolay Laptev

doi:10.1109/ICDMW.2015.104

Large-scale unusual time series detection

Rob J. Hyndman, Earo Wang, Nikolay Laptev

Econometrics & Business Statistics

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other

149 Citations (Scopus)

Abstract

It is becoming increasingly common for organizations to collect very large amounts of data over time, and to need to detect unusual or anomalous time series. For example, Yahoo has banks of mail servers that are monitored over time. Many measurements on server performance are collected every hour for each of thousands of servers. We wish to identify servers that are behaving unusually. We compute a vector of features on each time series, measuring characteristics of the series. The features may include lag correlation, strength of seasonality, spectral entropy, etc. Then we use a principal component decomposition on the features, and use various bivariate outlier detection methods applied to the first two principal components. This enables the most unusual series, based on their feature vectors, to be identified. The bivariate outlier detection methods used are based on highest density regions and α-hulls.

Original language	English
Title of host publication	Proceedings - 15th IEEE International Conference on Data Mining Workshop
Subtitle of host publication	ICDMW 2015
Editors	Peng Cui, Jennifer Dry, Charu Aggarwal, Zhi-Hua Zhou, Alexander Tuzhilin, Hui Xiong, Xindong Wu
Place of Publication	Piscataway NJ USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	1616-1619
Number of pages	4
ISBN (Print)	9781467384926
DOIs	https://doi.org/10.1109/ICDMW.2015.104
Publication status	Published - 2015
Event	IEEE International Conference on Data Mining Workshops 2015 - Bally's Atlantic City Hotel, Atlantic City, United States of America Duration: 14 Nov 2015 → 17 Nov 2015 Conference number: 15th https://icdm2015.stonybrook.edu/

Conference

Conference	IEEE International Conference on Data Mining Workshops 2015
Abbreviated title	ICDMW 2015
Country/Territory	United States of America
City	Atlantic City
Period	14/11/15 → 17/11/15
Internet address	https://icdm2015.stonybrook.edu/

Keywords

Feature Space
Multivariate Anomaly Detection
Outliers
Time Series Characteristics

Access to Document

10.1109/ICDMW.2015.104

Cite this

Hyndman, R. J., Wang, E., & Laptev, N. (2015). Large-scale unusual time series detection. In P. Cui, J. Dry, C. Aggarwal, Z.-H. Zhou, A. Tuzhilin, H. Xiong, & X. Wu (Eds.), Proceedings - 15th IEEE International Conference on Data Mining Workshop: ICDMW 2015 (pp. 1616-1619). Article 7395871 IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICDMW.2015.104

Hyndman, Rob J. ; Wang, Earo ; Laptev, Nikolay. / Large-scale unusual time series detection. Proceedings - 15th IEEE International Conference on Data Mining Workshop: ICDMW 2015. editor / Peng Cui ; Jennifer Dry ; Charu Aggarwal ; Zhi-Hua Zhou ; Alexander Tuzhilin ; Hui Xiong ; Xindong Wu. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2015. pp. 1616-1619

@inproceedings{a694128984db46eda9128af670fcab53,

title = "Large-scale unusual time series detection",

abstract = "It is becoming increasingly common for organizations to collect very large amounts of data over time, and to need to detect unusual or anomalous time series. For example, Yahoo has banks of mail servers that are monitored over time. Many measurements on server performance are collected every hour for each of thousands of servers. We wish to identify servers that are behaving unusually. We compute a vector of features on each time series, measuring characteristics of the series. The features may include lag correlation, strength of seasonality, spectral entropy, etc. Then we use a principal component decomposition on the features, and use various bivariate outlier detection methods applied to the first two principal components. This enables the most unusual series, based on their feature vectors, to be identified. The bivariate outlier detection methods used are based on highest density regions and α-hulls.",

keywords = "Feature Space, Multivariate Anomaly Detection, Outliers, Time Series Characteristics",

author = "Hyndman, {Rob J.} and Earo Wang and Nikolay Laptev",

year = "2015",

doi = "10.1109/ICDMW.2015.104",

language = "English",

isbn = "9781467384926",

pages = "1616--1619",

editor = "Peng Cui and Jennifer Dry and Charu Aggarwal and Zhi-Hua Zhou and Alexander Tuzhilin and Hui Xiong and Xindong Wu",

booktitle = "Proceedings - 15th IEEE International Conference on Data Mining Workshop",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

address = "United States of America",

note = "IEEE International Conference on Data Mining Workshops 2015, ICDMW 2015 ; Conference date: 14-11-2015 Through 17-11-2015",

url = "https://icdm2015.stonybrook.edu/",

}

Hyndman, RJ, Wang, E & Laptev, N 2015, Large-scale unusual time series detection. in P Cui, J Dry, C Aggarwal, Z-H Zhou, A Tuzhilin, H Xiong & X Wu (eds), Proceedings - 15th IEEE International Conference on Data Mining Workshop: ICDMW 2015., 7395871, IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 1616-1619, IEEE International Conference on Data Mining Workshops 2015, Atlantic City, New Jersey, United States of America, 14/11/15. https://doi.org/10.1109/ICDMW.2015.104

Large-scale unusual time series detection. / Hyndman, Rob J.; Wang, Earo; Laptev, Nikolay.
Proceedings - 15th IEEE International Conference on Data Mining Workshop: ICDMW 2015. ed. / Peng Cui; Jennifer Dry; Charu Aggarwal; Zhi-Hua Zhou; Alexander Tuzhilin; Hui Xiong; Xindong Wu. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers, 2015. p. 1616-1619 7395871.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other

TY - GEN

T1 - Large-scale unusual time series detection

AU - Hyndman, Rob J.

AU - Wang, Earo

AU - Laptev, Nikolay

N1 - Conference code: 15th

PY - 2015

Y1 - 2015

N2 - It is becoming increasingly common for organizations to collect very large amounts of data over time, and to need to detect unusual or anomalous time series. For example, Yahoo has banks of mail servers that are monitored over time. Many measurements on server performance are collected every hour for each of thousands of servers. We wish to identify servers that are behaving unusually. We compute a vector of features on each time series, measuring characteristics of the series. The features may include lag correlation, strength of seasonality, spectral entropy, etc. Then we use a principal component decomposition on the features, and use various bivariate outlier detection methods applied to the first two principal components. This enables the most unusual series, based on their feature vectors, to be identified. The bivariate outlier detection methods used are based on highest density regions and α-hulls.

AB - It is becoming increasingly common for organizations to collect very large amounts of data over time, and to need to detect unusual or anomalous time series. For example, Yahoo has banks of mail servers that are monitored over time. Many measurements on server performance are collected every hour for each of thousands of servers. We wish to identify servers that are behaving unusually. We compute a vector of features on each time series, measuring characteristics of the series. The features may include lag correlation, strength of seasonality, spectral entropy, etc. Then we use a principal component decomposition on the features, and use various bivariate outlier detection methods applied to the first two principal components. This enables the most unusual series, based on their feature vectors, to be identified. The bivariate outlier detection methods used are based on highest density regions and α-hulls.

KW - Feature Space

KW - Multivariate Anomaly Detection

KW - Outliers

KW - Time Series Characteristics

UR - http://www.scopus.com/inward/record.url?scp=84964715434&partnerID=8YFLogxK

U2 - 10.1109/ICDMW.2015.104

DO - 10.1109/ICDMW.2015.104

M3 - Conference Paper

AN - SCOPUS:84964715434

SN - 9781467384926

SP - 1616

EP - 1619

BT - Proceedings - 15th IEEE International Conference on Data Mining Workshop

A2 - Cui, Peng

A2 - Dry, Jennifer

A2 - Aggarwal, Charu

A2 - Zhou, Zhi-Hua

A2 - Tuzhilin, Alexander

A2 - Xiong, Hui

A2 - Wu, Xindong

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

T2 - IEEE International Conference on Data Mining Workshops 2015

Y2 - 14 November 2015 through 17 November 2015

ER -

Large-scale unusual time series detection

Abstract

Conference

Keywords

Access to Document

Other files and links

Cite this