Analyzing concept drift and shift from sample data

Geoffrey I. Webb, Loong Kuan Lee, Bart Goethals, François Petitjean

    Research output: Contribution to journalArticleResearchpeer-review

    19 Citations (Scopus)

    Abstract

    Concept drift and shift are major issues that greatly affect the accuracy and reliability of many real-world applications of machine learning. We propose a new data mining task, concept drift mapping—the description and analysis of instances of concept drift or shift. We argue that concept drift mapping is an essential prerequisite for tackling concept drift and shift. We propose tools for this purpose, arguing for the importance of quantitative descriptions of drift and shift in marginal distributions. We present quantitative concept drift mapping techniques, along with methods for visualizing their results. We illustrate their effectiveness for real-world applications across energy-pricing, vegetation monitoring and airline scheduling.

    Original languageEnglish
    Pages (from-to)1179-1199
    Number of pages21
    JournalData Mining and Knowledge Discovery
    Volume32
    Issue number5
    DOIs
    Publication statusPublished - Sep 2018

    Keywords

    • Concept drift
    • Concept shift
    • Mapping
    • Non-stationary distribution
    • Visualisation

    Cite this