Exceptional contrast set mining: moving beyond the deluge of the obvious

Dang Nguyen, Wei Luo, Dinh Phung, Svetha Venkatesh

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

3 Citations (Scopus)

Abstract

Data scientists, with access to fast growing data and computing power, constantly look for algorithms with greater detection power to discover “novel” knowledge. But more often than not, their algorithms give them too many outputs that are either highly speculative or simply confirming what the domain experts already know. To escape this dilemma, we need algorithms that move beyond the obvious association analyses and leverage domain analytic objectives (aka. KPIs) to look for higher order connections. We propose a new technique Exceptional Contrast Set Mining that first gathers a succinct collection of affirmative contrast sets based on the principle of redundant information elimination. Then it discovers exceptional contrast sets that contradict the affirmative contrast sets. The algorithm has been successfully applied to several analytic consulting projects. In particular, during an analysis of a state-wide cancer registry, it discovered a surprising regional difference in breast cancer screening.

Original languageEnglish
Title of host publicationAI 2016
Subtitle of host publicationAdvances in Artificial Intelligence - 29th Australasian Joint Conference Hobart, TAS, Australia, December 5–8, 2016 Proceedings
EditorsByeong Ho Kang, Quan Bai
Place of PublicationCham Switzerland
PublisherSpringer
Pages455-468
Number of pages14
ISBN (Electronic)9783319501277
ISBN (Print)9783319501260
DOIs
Publication statusPublished - 2016
Externally publishedYes
EventAustralasian Joint Conference on Artificial Intelligence 2016 - Hobart, Australia
Duration: 5 Dec 20168 Dec 2016
Conference number: 29th
https://ai2016.net/
https://link.springer.com/book/10.1007/978-3-319-50127-7 (Proceedings)

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume9992
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceAustralasian Joint Conference on Artificial Intelligence 2016
Abbreviated titleAI 2016
CountryAustralia
CityHobart
Period5/12/168/12/16
Internet address

Cite this