A relevance weighted ensemble model for anomaly detection in switching data streams

Mahsa Salehi, Christopher A. Leckie, Masud Moshtaghi, Tharshan Vaithianathan

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    Abstract

    Anomaly detection in data streams plays a vital role in online data mining applications. A major challenge for anomaly detection is the dynamically changing nature of many monitoring environments. This causes a problem for traditional anomaly detection techniques in data streams, which assume a relatively static monitoring environment. In an environment that is intermittently changing (known as switching data streams), static approaches can yield a high error rate in terms of false positives. To cope with dynamic environments, we require an approach that can learn from the history of normal behaviour in data streams, while accounting for the fact that not all time periods in the past are equally relevant. Consequently, we have proposed a relevance-weighted ensemble model for learning normal behaviour, which forms the basis of our anomaly detection scheme. The advantage of this approach is that it can improve the accuracy of detection by using relevant history, while remaining computationally efficient. Our solution provides a novel contribution through the use of ensemble techniques for anomaly detection in switching data streams. Our empirical results on real and synthetic data streams show that we can achieve substantial improvements compared to a recent anomaly detection algorithm for data streams.
    Original languageEnglish
    Title of host publicationAdvances in Knowledge Discovery and Data Mining: 18th Pacific-Asia Conference (Proceedings (PAKDD 2014)
    Subtitle of host publicationTainan, Taiwan, May 13-16, 2014 Proceedings, Part II
    EditorsVincent S. Tseng, Tu Bao Ho, Zhi-Hua Zhou, Arbee L. P. Chen, Hung-Yu Kao
    Place of PublicationCham [Switzerland]
    PublisherSpringer
    Pages461 - 473
    Number of pages13
    ISBN (Electronic)9783319066059
    ISBN (Print)9783319066042
    DOIs
    Publication statusPublished - 2014
    EventPacific-Asia Conference on Knowledge Discovery and Data Mining 2014 - Tainan, Taiwan
    Duration: 13 May 201416 May 2014
    Conference number: 18th
    https://sites.google.com/site/pakdd2014/

    Conference

    ConferencePacific-Asia Conference on Knowledge Discovery and Data Mining 2014
    Abbreviated titlePAKDD 2014
    CountryTaiwan
    CityTainan
    Period13/05/1416/05/14
    Internet address

    Keywords

    • Anomaly detection
    • Ensemble models
    • Data streams

    Cite this

    Salehi, M., Leckie, C. A., Moshtaghi, M., & Vaithianathan, T. (2014). A relevance weighted ensemble model for anomaly detection in switching data streams. In V. S. Tseng, T. B. Ho, Z-H. Zhou, A. L. P. Chen, & H-Y. Kao (Eds.), Advances in Knowledge Discovery and Data Mining: 18th Pacific-Asia Conference (Proceedings (PAKDD 2014): Tainan, Taiwan, May 13-16, 2014 Proceedings, Part II (pp. 461 - 473). Cham [Switzerland]: Springer. https://doi.org/10.1007/978-3-319-06605-9_38
    Salehi, Mahsa ; Leckie, Christopher A. ; Moshtaghi, Masud ; Vaithianathan, Tharshan. / A relevance weighted ensemble model for anomaly detection in switching data streams. Advances in Knowledge Discovery and Data Mining: 18th Pacific-Asia Conference (Proceedings (PAKDD 2014): Tainan, Taiwan, May 13-16, 2014 Proceedings, Part II. editor / Vincent S. Tseng ; Tu Bao Ho ; Zhi-Hua Zhou ; Arbee L. P. Chen ; Hung-Yu Kao. Cham [Switzerland] : Springer, 2014. pp. 461 - 473
    @inproceedings{1569796fb58244b5bf2317423d69339f,
    title = "A relevance weighted ensemble model for anomaly detection in switching data streams",
    abstract = "Anomaly detection in data streams plays a vital role in online data mining applications. A major challenge for anomaly detection is the dynamically changing nature of many monitoring environments. This causes a problem for traditional anomaly detection techniques in data streams, which assume a relatively static monitoring environment. In an environment that is intermittently changing (known as switching data streams), static approaches can yield a high error rate in terms of false positives. To cope with dynamic environments, we require an approach that can learn from the history of normal behaviour in data streams, while accounting for the fact that not all time periods in the past are equally relevant. Consequently, we have proposed a relevance-weighted ensemble model for learning normal behaviour, which forms the basis of our anomaly detection scheme. The advantage of this approach is that it can improve the accuracy of detection by using relevant history, while remaining computationally efficient. Our solution provides a novel contribution through the use of ensemble techniques for anomaly detection in switching data streams. Our empirical results on real and synthetic data streams show that we can achieve substantial improvements compared to a recent anomaly detection algorithm for data streams.",
    keywords = "Anomaly detection, Ensemble models, Data streams",
    author = "Mahsa Salehi and Leckie, {Christopher A.} and Masud Moshtaghi and Tharshan Vaithianathan",
    year = "2014",
    doi = "10.1007/978-3-319-06605-9_38",
    language = "English",
    isbn = "9783319066042",
    pages = "461 -- 473",
    editor = "Tseng, {Vincent S.} and Ho, {Tu Bao} and Zhi-Hua Zhou and Chen, {Arbee L. P.} and Hung-Yu Kao",
    booktitle = "Advances in Knowledge Discovery and Data Mining: 18th Pacific-Asia Conference (Proceedings (PAKDD 2014)",
    publisher = "Springer",

    }

    Salehi, M, Leckie, CA, Moshtaghi, M & Vaithianathan, T 2014, A relevance weighted ensemble model for anomaly detection in switching data streams. in VS Tseng, TB Ho, Z-H Zhou, ALP Chen & H-Y Kao (eds), Advances in Knowledge Discovery and Data Mining: 18th Pacific-Asia Conference (Proceedings (PAKDD 2014): Tainan, Taiwan, May 13-16, 2014 Proceedings, Part II. Springer, Cham [Switzerland], pp. 461 - 473, Pacific-Asia Conference on Knowledge Discovery and Data Mining 2014, Tainan, Taiwan, 13/05/14. https://doi.org/10.1007/978-3-319-06605-9_38

    A relevance weighted ensemble model for anomaly detection in switching data streams. / Salehi, Mahsa; Leckie, Christopher A.; Moshtaghi, Masud; Vaithianathan, Tharshan.

    Advances in Knowledge Discovery and Data Mining: 18th Pacific-Asia Conference (Proceedings (PAKDD 2014): Tainan, Taiwan, May 13-16, 2014 Proceedings, Part II. ed. / Vincent S. Tseng; Tu Bao Ho; Zhi-Hua Zhou; Arbee L. P. Chen; Hung-Yu Kao. Cham [Switzerland] : Springer, 2014. p. 461 - 473.

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    TY - GEN

    T1 - A relevance weighted ensemble model for anomaly detection in switching data streams

    AU - Salehi, Mahsa

    AU - Leckie, Christopher A.

    AU - Moshtaghi, Masud

    AU - Vaithianathan, Tharshan

    PY - 2014

    Y1 - 2014

    N2 - Anomaly detection in data streams plays a vital role in online data mining applications. A major challenge for anomaly detection is the dynamically changing nature of many monitoring environments. This causes a problem for traditional anomaly detection techniques in data streams, which assume a relatively static monitoring environment. In an environment that is intermittently changing (known as switching data streams), static approaches can yield a high error rate in terms of false positives. To cope with dynamic environments, we require an approach that can learn from the history of normal behaviour in data streams, while accounting for the fact that not all time periods in the past are equally relevant. Consequently, we have proposed a relevance-weighted ensemble model for learning normal behaviour, which forms the basis of our anomaly detection scheme. The advantage of this approach is that it can improve the accuracy of detection by using relevant history, while remaining computationally efficient. Our solution provides a novel contribution through the use of ensemble techniques for anomaly detection in switching data streams. Our empirical results on real and synthetic data streams show that we can achieve substantial improvements compared to a recent anomaly detection algorithm for data streams.

    AB - Anomaly detection in data streams plays a vital role in online data mining applications. A major challenge for anomaly detection is the dynamically changing nature of many monitoring environments. This causes a problem for traditional anomaly detection techniques in data streams, which assume a relatively static monitoring environment. In an environment that is intermittently changing (known as switching data streams), static approaches can yield a high error rate in terms of false positives. To cope with dynamic environments, we require an approach that can learn from the history of normal behaviour in data streams, while accounting for the fact that not all time periods in the past are equally relevant. Consequently, we have proposed a relevance-weighted ensemble model for learning normal behaviour, which forms the basis of our anomaly detection scheme. The advantage of this approach is that it can improve the accuracy of detection by using relevant history, while remaining computationally efficient. Our solution provides a novel contribution through the use of ensemble techniques for anomaly detection in switching data streams. Our empirical results on real and synthetic data streams show that we can achieve substantial improvements compared to a recent anomaly detection algorithm for data streams.

    KW - Anomaly detection

    KW - Ensemble models

    KW - Data streams

    UR - http://www.scopus.com/inward/record.url?scp=84901271058&partnerID=8YFLogxK

    U2 - 10.1007/978-3-319-06605-9_38

    DO - 10.1007/978-3-319-06605-9_38

    M3 - Conference Paper

    SN - 9783319066042

    SP - 461

    EP - 473

    BT - Advances in Knowledge Discovery and Data Mining: 18th Pacific-Asia Conference (Proceedings (PAKDD 2014)

    A2 - Tseng, Vincent S.

    A2 - Ho, Tu Bao

    A2 - Zhou, Zhi-Hua

    A2 - Chen, Arbee L. P.

    A2 - Kao, Hung-Yu

    PB - Springer

    CY - Cham [Switzerland]

    ER -

    Salehi M, Leckie CA, Moshtaghi M, Vaithianathan T. A relevance weighted ensemble model for anomaly detection in switching data streams. In Tseng VS, Ho TB, Zhou Z-H, Chen ALP, Kao H-Y, editors, Advances in Knowledge Discovery and Data Mining: 18th Pacific-Asia Conference (Proceedings (PAKDD 2014): Tainan, Taiwan, May 13-16, 2014 Proceedings, Part II. Cham [Switzerland]: Springer. 2014. p. 461 - 473 https://doi.org/10.1007/978-3-319-06605-9_38