Fast memory efficient local outlier detection in data streams

Mahsa Salehi, Christopher Leckie, James C. Bezdek, Tharshan Vaithianathan, Xuyun Zhang

Research output: Chapter in Book/Report/Conference proceedingConference PaperOtherpeer-review

Abstract

Outlier detection is an important task in data mining. With the growing need to analyze high speed data streams, the task of outlier detection becomes even more challenging as traditional outlier detection techniques can no longer assume that all the data can be stored for processing. While the wellknown Local Outlier Factor (LOF) algorithm has an incremental version (called iLOF), it assumes unbounded memory to keep all previous data points. In this paper, we propose a memory efficient incremental local outlier (MiLOF) detection algorithm for data streams, and a more flexible version (MiLOF F), both have an accuracy close to iLOF but within a fixed memory bound. In addition MiLOF F is robust to changes in the number of data points, underlying clusters and dimensions in the data stream.

Original languageEnglish
Title of host publicationProceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017)
Subtitle of host publicationSan Diego, California, USA, 19-22 April 2017
EditorsYannis Papakonstantinou, Yanlei Diao
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages51-52
Number of pages2
ISBN (Electronic)9781509065431
ISBN (Print)9781509065448
DOIs
Publication statusPublished - 16 May 2017
Externally publishedYes
EventIEEE International Conference on Data Engineering 2017 - Hilton San Diego Resort and Spa in Mission Bay, San Diego, United States of America
Duration: 19 Apr 201722 Apr 2017
Conference number: 33rd
http://icde2017.sdsc.edu/ (Conference website)

Conference

ConferenceIEEE International Conference on Data Engineering 2017
Abbreviated titleICDE 2017
CountryUnited States of America
CitySan Diego
Period19/04/1722/04/17
Internet address

Cite this

Salehi, M., Leckie, C., Bezdek, J. C., Vaithianathan, T., & Zhang, X. (2017). Fast memory efficient local outlier detection in data streams. In Y. Papakonstantinou, & Y. Diao (Eds.), Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017 (pp. 51-52). [7929928] IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICDE.2017.32
Salehi, Mahsa ; Leckie, Christopher ; Bezdek, James C. ; Vaithianathan, Tharshan ; Zhang, Xuyun. / Fast memory efficient local outlier detection in data streams. Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017. editor / Yannis Papakonstantinou ; Yanlei Diao. IEEE, Institute of Electrical and Electronics Engineers, 2017. pp. 51-52
@inproceedings{3ebc151372d44b47bfb9560468036911,
title = "Fast memory efficient local outlier detection in data streams",
abstract = "Outlier detection is an important task in data mining. With the growing need to analyze high speed data streams, the task of outlier detection becomes even more challenging as traditional outlier detection techniques can no longer assume that all the data can be stored for processing. While the wellknown Local Outlier Factor (LOF) algorithm has an incremental version (called iLOF), it assumes unbounded memory to keep all previous data points. In this paper, we propose a memory efficient incremental local outlier (MiLOF) detection algorithm for data streams, and a more flexible version (MiLOF F), both have an accuracy close to iLOF but within a fixed memory bound. In addition MiLOF F is robust to changes in the number of data points, underlying clusters and dimensions in the data stream.",
author = "Mahsa Salehi and Christopher Leckie and Bezdek, {James C.} and Tharshan Vaithianathan and Xuyun Zhang",
year = "2017",
month = "5",
day = "16",
doi = "10.1109/ICDE.2017.32",
language = "English",
isbn = "9781509065448",
pages = "51--52",
editor = "Yannis Papakonstantinou and Yanlei Diao",
booktitle = "Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017)",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
address = "United States of America",

}

Salehi, M, Leckie, C, Bezdek, JC, Vaithianathan, T & Zhang, X 2017, Fast memory efficient local outlier detection in data streams. in Y Papakonstantinou & Y Diao (eds), Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017., 7929928, IEEE, Institute of Electrical and Electronics Engineers, pp. 51-52, IEEE International Conference on Data Engineering 2017, San Diego, United States of America, 19/04/17. https://doi.org/10.1109/ICDE.2017.32

Fast memory efficient local outlier detection in data streams. / Salehi, Mahsa; Leckie, Christopher; Bezdek, James C.; Vaithianathan, Tharshan; Zhang, Xuyun.

Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017. ed. / Yannis Papakonstantinou; Yanlei Diao. IEEE, Institute of Electrical and Electronics Engineers, 2017. p. 51-52 7929928.

Research output: Chapter in Book/Report/Conference proceedingConference PaperOtherpeer-review

TY - GEN

T1 - Fast memory efficient local outlier detection in data streams

AU - Salehi, Mahsa

AU - Leckie, Christopher

AU - Bezdek, James C.

AU - Vaithianathan, Tharshan

AU - Zhang, Xuyun

PY - 2017/5/16

Y1 - 2017/5/16

N2 - Outlier detection is an important task in data mining. With the growing need to analyze high speed data streams, the task of outlier detection becomes even more challenging as traditional outlier detection techniques can no longer assume that all the data can be stored for processing. While the wellknown Local Outlier Factor (LOF) algorithm has an incremental version (called iLOF), it assumes unbounded memory to keep all previous data points. In this paper, we propose a memory efficient incremental local outlier (MiLOF) detection algorithm for data streams, and a more flexible version (MiLOF F), both have an accuracy close to iLOF but within a fixed memory bound. In addition MiLOF F is robust to changes in the number of data points, underlying clusters and dimensions in the data stream.

AB - Outlier detection is an important task in data mining. With the growing need to analyze high speed data streams, the task of outlier detection becomes even more challenging as traditional outlier detection techniques can no longer assume that all the data can be stored for processing. While the wellknown Local Outlier Factor (LOF) algorithm has an incremental version (called iLOF), it assumes unbounded memory to keep all previous data points. In this paper, we propose a memory efficient incremental local outlier (MiLOF) detection algorithm for data streams, and a more flexible version (MiLOF F), both have an accuracy close to iLOF but within a fixed memory bound. In addition MiLOF F is robust to changes in the number of data points, underlying clusters and dimensions in the data stream.

UR - http://www.scopus.com/inward/record.url?scp=85021238905&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2017.32

DO - 10.1109/ICDE.2017.32

M3 - Conference Paper

SN - 9781509065448

SP - 51

EP - 52

BT - Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017)

A2 - Papakonstantinou, Yannis

A2 - Diao, Yanlei

PB - IEEE, Institute of Electrical and Electronics Engineers

ER -

Salehi M, Leckie C, Bezdek JC, Vaithianathan T, Zhang X. Fast memory efficient local outlier detection in data streams. In Papakonstantinou Y, Diao Y, editors, Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017. IEEE, Institute of Electrical and Electronics Engineers. 2017. p. 51-52. 7929928 https://doi.org/10.1109/ICDE.2017.32