Fast memory efficient local outlier detection in data streams

Mahsa Salehi; Christopher Leckie; James C. Bezdek; Tharshan Vaithianathan; Xuyun Zhang

doi:10.1109/ICDE.2017.32

Fast memory efficient local outlier detection in data streams

Mahsa Salehi
, Christopher Leckie
, James C. Bezdek
, Tharshan Vaithianathan
, Xuyun Zhang

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other › peer-review

6 Citations (Scopus)

Abstract

Outlier detection is an important task in data mining. With the growing need to analyze high speed data streams, the task of outlier detection becomes even more challenging as traditional outlier detection techniques can no longer assume that all the data can be stored for processing. While the wellknown Local Outlier Factor (LOF) algorithm has an incremental version (called iLOF), it assumes unbounded memory to keep all previous data points. In this paper, we propose a memory efficient incremental local outlier (MiLOF) detection algorithm for data streams, and a more flexible version (MiLOF F), both have an accuracy close to iLOF but within a fixed memory bound. In addition MiLOF F is robust to changes in the number of data points, underlying clusters and dimensions in the data stream.

Original language	English
Title of host publication	Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017)
Subtitle of host publication	San Diego, California, USA, 19-22 April 2017
Editors	Yannis Papakonstantinou, Yanlei Diao
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	51-52
Number of pages	2
ISBN (Electronic)	9781509065431
ISBN (Print)	9781509065448
DOIs	https://doi.org/10.1109/ICDE.2017.32
Publication status	Published - 16 May 2017
Externally published	Yes
Event	IEEE International Conference on Data Engineering 2017 - Hilton San Diego Resort and Spa in Mission Bay, San Diego, United States of America Duration: 19 Apr 2017 → 22 Apr 2017 Conference number: 33rd http://icde2017.sdsc.edu/ (Conference website) https://ieeexplore.ieee.org/xpl/conhome/7929494/proceeding (Proceedings)

Conference

Conference	IEEE International Conference on Data Engineering 2017
Abbreviated title	ICDE 2017
Country/Territory	United States of America
City	San Diego
Period	19/04/17 → 22/04/17
Internet address	http://icde2017.sdsc.edu/ (Conference website) https://ieeexplore.ieee.org/xpl/conhome/7929494/proceeding (Proceedings)

Access to Document

10.1109/ICDE.2017.32

Cite this

Salehi, M., Leckie, C., Bezdek, J. C., Vaithianathan, T., & Zhang, X. (2017). Fast memory efficient local outlier detection in data streams. In Y. Papakonstantinou, & Y. Diao (Eds.), Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017 (pp. 51-52). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICDE.2017.32

Salehi, Mahsa ; Leckie, Christopher ; Bezdek, James C. et al. / Fast memory efficient local outlier detection in data streams. Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017. editor / Yannis Papakonstantinou ; Yanlei Diao. IEEE, Institute of Electrical and Electronics Engineers, 2017. pp. 51-52

@inproceedings{3ebc151372d44b47bfb9560468036911,

title = "Fast memory efficient local outlier detection in data streams",

abstract = "Outlier detection is an important task in data mining. With the growing need to analyze high speed data streams, the task of outlier detection becomes even more challenging as traditional outlier detection techniques can no longer assume that all the data can be stored for processing. While the wellknown Local Outlier Factor (LOF) algorithm has an incremental version (called iLOF), it assumes unbounded memory to keep all previous data points. In this paper, we propose a memory efficient incremental local outlier (MiLOF) detection algorithm for data streams, and a more flexible version (MiLOF F), both have an accuracy close to iLOF but within a fixed memory bound. In addition MiLOF F is robust to changes in the number of data points, underlying clusters and dimensions in the data stream.",

author = "Mahsa Salehi and Christopher Leckie and Bezdek, \{James C.\} and Tharshan Vaithianathan and Xuyun Zhang",

year = "2017",

month = may,

day = "16",

doi = "10.1109/ICDE.2017.32",

language = "English",

isbn = "9781509065448",

pages = "51--52",

editor = "Yannis Papakonstantinou and Yanlei Diao",

booktitle = "Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017)",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

address = "United States of America",

note = "IEEE International Conference on Data Engineering 2017, ICDE 2017 ; Conference date: 19-04-2017 Through 22-04-2017",

url = "http://icde2017.sdsc.edu/, https://ieeexplore.ieee.org/xpl/conhome/7929494/proceeding",

}

Salehi, M, Leckie, C, Bezdek, JC, Vaithianathan, T & Zhang, X 2017, Fast memory efficient local outlier detection in data streams. in Y Papakonstantinou & Y Diao (eds), Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017. IEEE, Institute of Electrical and Electronics Engineers, pp. 51-52, IEEE International Conference on Data Engineering 2017, San Diego, California, United States of America, 19/04/17. https://doi.org/10.1109/ICDE.2017.32

Fast memory efficient local outlier detection in data streams. / Salehi, Mahsa; Leckie, Christopher; Bezdek, James C. et al.
Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017. ed. / Yannis Papakonstantinou; Yanlei Diao. IEEE, Institute of Electrical and Electronics Engineers, 2017. p. 51-52.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other › peer-review

TY - GEN

T1 - Fast memory efficient local outlier detection in data streams

AU - Salehi, Mahsa

AU - Leckie, Christopher

AU - Bezdek, James C.

AU - Vaithianathan, Tharshan

AU - Zhang, Xuyun

N1 - Conference code: 33rd

PY - 2017/5/16

Y1 - 2017/5/16

N2 - Outlier detection is an important task in data mining. With the growing need to analyze high speed data streams, the task of outlier detection becomes even more challenging as traditional outlier detection techniques can no longer assume that all the data can be stored for processing. While the wellknown Local Outlier Factor (LOF) algorithm has an incremental version (called iLOF), it assumes unbounded memory to keep all previous data points. In this paper, we propose a memory efficient incremental local outlier (MiLOF) detection algorithm for data streams, and a more flexible version (MiLOF F), both have an accuracy close to iLOF but within a fixed memory bound. In addition MiLOF F is robust to changes in the number of data points, underlying clusters and dimensions in the data stream.

AB - Outlier detection is an important task in data mining. With the growing need to analyze high speed data streams, the task of outlier detection becomes even more challenging as traditional outlier detection techniques can no longer assume that all the data can be stored for processing. While the wellknown Local Outlier Factor (LOF) algorithm has an incremental version (called iLOF), it assumes unbounded memory to keep all previous data points. In this paper, we propose a memory efficient incremental local outlier (MiLOF) detection algorithm for data streams, and a more flexible version (MiLOF F), both have an accuracy close to iLOF but within a fixed memory bound. In addition MiLOF F is robust to changes in the number of data points, underlying clusters and dimensions in the data stream.

UR - https://www.scopus.com/pages/publications/85021238905

U2 - 10.1109/ICDE.2017.32

DO - 10.1109/ICDE.2017.32

M3 - Conference Paper

AN - SCOPUS:85021238905

SN - 9781509065448

SP - 51

EP - 52

BT - Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017)

A2 - Papakonstantinou, Yannis

A2 - Diao, Yanlei

PB - IEEE, Institute of Electrical and Electronics Engineers

T2 - IEEE International Conference on Data Engineering 2017

Y2 - 19 April 2017 through 22 April 2017

ER -

Salehi M, Leckie C, Bezdek JC, Vaithianathan T, Zhang X. Fast memory efficient local outlier detection in data streams. In Papakonstantinou Y, Diao Y, editors, Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE 2017): San Diego, California, USA, 19-22 April 2017. IEEE, Institute of Electrical and Electronics Engineers. 2017. p. 51-52 doi: 10.1109/ICDE.2017.32

Fast memory efficient local outlier detection in data streams

Abstract

Conference

Access to Document

Other files and links

Research output

Fast memory efficient local outlier detection in data streams

Cite this