Analyzing concept drift and shift from sample data

Geoffrey I. Webb, Loong Kuan Lee, Bart Goethals, François Petitjean

Research output: Contribution to journalArticleResearchpeer-review

86 Citations (Scopus)

Abstract

Concept drift and shift are major issues that greatly affect the accuracy and reliability of many real-world applications of machine learning. We propose a new data mining task, concept drift mapping—the description and analysis of instances of concept drift or shift. We argue that concept drift mapping is an essential prerequisite for tackling concept drift and shift. We propose tools for this purpose, arguing for the importance of quantitative descriptions of drift and shift in marginal distributions. We present quantitative concept drift mapping techniques, along with methods for visualizing their results. We illustrate their effectiveness for real-world applications across energy-pricing, vegetation monitoring and airline scheduling.

Original languageEnglish
Pages (from-to)1179-1199
Number of pages21
JournalData Mining and Knowledge Discovery
Volume32
Issue number5
DOIs
Publication statusPublished - Sept 2018

Keywords

  • Concept drift
  • Concept shift
  • Mapping
  • Non-stationary distribution
  • Visualisation

Cite this