A gradient-based algorithm for trend and outlier prediction in dynamic data streams

Dawei Sun, Vincent CS Lee, Ye Lu

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Trend and outlier are frequently used to derive early warning predictive signal to decision maker in order to achieve ultimate quality decision outcome in domain specific (e.g. commercial, scientific, biomedical and engineering, just to name a few) applications. We develop a gradient-based algorithm using sample entropy gradient(SEG) for trend and outlier prediction in high frequency time series data streams. L2 similarity measure (Euclidean distance between two linearized gradient curves is then computed and used to quantify the degree of similarity and compared with a threshold L2 value to judge the extend of dissimilarity that would be classified as outlier. SEG algorithm which circumvents the need to pre-specify tolerance parameter in those cross sample entropy (CSE)-based algorithms that invariably involve real domain expert to set the tolerance threshold. We conduct real data experiments on SEG algorithm to two application areas: dynamic wind speed data stream; and financial time series data. Our experiments demonstrated that SEG algorithm can be feasibly used in online implementation to derive predictive early warning signals to domain-specific decision maker.

Original languageEnglish
Title of host publicationProceedings of the 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA 2017)
Subtitle of host publication18 – 20 June 2017 Siem Reap, Cambodia
EditorsXing Zhu
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages1978-1983
Number of pages6
ISBN (Electronic)9781538621035, 9781509061617
ISBN (Print)9781509061624
DOIs
Publication statusPublished - 8 Feb 2018
EventIEEE Conference on Industrial Electronics and Applications 2017 - Siem Reap, Cambodia
Duration: 18 Jun 201720 Jun 2017
Conference number: 12th
http://www.ieeeiciea.org/2017/

Conference

ConferenceIEEE Conference on Industrial Electronics and Applications 2017
Abbreviated titleICIEA 2017
CountryCambodia
CitySiem Reap
Period18/06/1720/06/17
Internet address

Keywords

  • Correlations
  • Cross-sample Entropy
  • Sample Entropy Gradient
  • Similarity
  • Time-series data

Cite this

Sun, D., Lee, V. CS., & Lu, Y. (2018). A gradient-based algorithm for trend and outlier prediction in dynamic data streams. In X. Zhu (Ed.), Proceedings of the 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA 2017): 18 – 20 June 2017 Siem Reap, Cambodia (pp. 1978-1983). Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICIEA.2017.8283162
Sun, Dawei ; Lee, Vincent CS ; Lu, Ye. / A gradient-based algorithm for trend and outlier prediction in dynamic data streams. Proceedings of the 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA 2017): 18 – 20 June 2017 Siem Reap, Cambodia. editor / Xing Zhu. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. pp. 1978-1983
@inproceedings{b79679303af74bf09266529f6e691930,
title = "A gradient-based algorithm for trend and outlier prediction in dynamic data streams",
abstract = "Trend and outlier are frequently used to derive early warning predictive signal to decision maker in order to achieve ultimate quality decision outcome in domain specific (e.g. commercial, scientific, biomedical and engineering, just to name a few) applications. We develop a gradient-based algorithm using sample entropy gradient(SEG) for trend and outlier prediction in high frequency time series data streams. L2 similarity measure (Euclidean distance between two linearized gradient curves is then computed and used to quantify the degree of similarity and compared with a threshold L2 value to judge the extend of dissimilarity that would be classified as outlier. SEG algorithm which circumvents the need to pre-specify tolerance parameter in those cross sample entropy (CSE)-based algorithms that invariably involve real domain expert to set the tolerance threshold. We conduct real data experiments on SEG algorithm to two application areas: dynamic wind speed data stream; and financial time series data. Our experiments demonstrated that SEG algorithm can be feasibly used in online implementation to derive predictive early warning signals to domain-specific decision maker.",
keywords = "Correlations, Cross-sample Entropy, Sample Entropy Gradient, Similarity, Time-series data",
author = "Dawei Sun and Lee, {Vincent CS} and Ye Lu",
year = "2018",
month = "2",
day = "8",
doi = "10.1109/ICIEA.2017.8283162",
language = "English",
isbn = "9781509061624",
pages = "1978--1983",
editor = "Zhu, {Xing }",
booktitle = "Proceedings of the 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA 2017)",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
address = "United States of America",

}

Sun, D, Lee, VCS & Lu, Y 2018, A gradient-based algorithm for trend and outlier prediction in dynamic data streams. in X Zhu (ed.), Proceedings of the 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA 2017): 18 – 20 June 2017 Siem Reap, Cambodia. IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 1978-1983, IEEE Conference on Industrial Electronics and Applications 2017, Siem Reap, Cambodia, 18/06/17. https://doi.org/10.1109/ICIEA.2017.8283162

A gradient-based algorithm for trend and outlier prediction in dynamic data streams. / Sun, Dawei; Lee, Vincent CS; Lu, Ye.

Proceedings of the 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA 2017): 18 – 20 June 2017 Siem Reap, Cambodia. ed. / Xing Zhu. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. p. 1978-1983.

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - A gradient-based algorithm for trend and outlier prediction in dynamic data streams

AU - Sun, Dawei

AU - Lee, Vincent CS

AU - Lu, Ye

PY - 2018/2/8

Y1 - 2018/2/8

N2 - Trend and outlier are frequently used to derive early warning predictive signal to decision maker in order to achieve ultimate quality decision outcome in domain specific (e.g. commercial, scientific, biomedical and engineering, just to name a few) applications. We develop a gradient-based algorithm using sample entropy gradient(SEG) for trend and outlier prediction in high frequency time series data streams. L2 similarity measure (Euclidean distance between two linearized gradient curves is then computed and used to quantify the degree of similarity and compared with a threshold L2 value to judge the extend of dissimilarity that would be classified as outlier. SEG algorithm which circumvents the need to pre-specify tolerance parameter in those cross sample entropy (CSE)-based algorithms that invariably involve real domain expert to set the tolerance threshold. We conduct real data experiments on SEG algorithm to two application areas: dynamic wind speed data stream; and financial time series data. Our experiments demonstrated that SEG algorithm can be feasibly used in online implementation to derive predictive early warning signals to domain-specific decision maker.

AB - Trend and outlier are frequently used to derive early warning predictive signal to decision maker in order to achieve ultimate quality decision outcome in domain specific (e.g. commercial, scientific, biomedical and engineering, just to name a few) applications. We develop a gradient-based algorithm using sample entropy gradient(SEG) for trend and outlier prediction in high frequency time series data streams. L2 similarity measure (Euclidean distance between two linearized gradient curves is then computed and used to quantify the degree of similarity and compared with a threshold L2 value to judge the extend of dissimilarity that would be classified as outlier. SEG algorithm which circumvents the need to pre-specify tolerance parameter in those cross sample entropy (CSE)-based algorithms that invariably involve real domain expert to set the tolerance threshold. We conduct real data experiments on SEG algorithm to two application areas: dynamic wind speed data stream; and financial time series data. Our experiments demonstrated that SEG algorithm can be feasibly used in online implementation to derive predictive early warning signals to domain-specific decision maker.

KW - Correlations

KW - Cross-sample Entropy

KW - Sample Entropy Gradient

KW - Similarity

KW - Time-series data

UR - http://www.scopus.com/inward/record.url?scp=85047496241&partnerID=8YFLogxK

U2 - 10.1109/ICIEA.2017.8283162

DO - 10.1109/ICIEA.2017.8283162

M3 - Conference Paper

SN - 9781509061624

SP - 1978

EP - 1983

BT - Proceedings of the 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA 2017)

A2 - Zhu, Xing

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

ER -

Sun D, Lee VCS, Lu Y. A gradient-based algorithm for trend and outlier prediction in dynamic data streams. In Zhu X, editor, Proceedings of the 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA 2017): 18 – 20 June 2017 Siem Reap, Cambodia. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2018. p. 1978-1983 https://doi.org/10.1109/ICIEA.2017.8283162