Stabilizing linear prediction models using autoencoder

Shivapratap Gopakumar, Truyen Tran, Dinh Phung, Svetha Venkatesh

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

Abstract

To date, the instability of prognostic predictors in a sparse high dimensional model, which hinders their clinical adoption, has received little attention. Stable prediction is often overlooked in favour of performance. Yet, stability prevails as key when adopting models in critical areas as healthcare. Our study proposes a stabilization scheme by detecting higher order feature correlations. Using a linear model as basis for prediction, we achieve feature stability by regularizing latent correlation in features. Latent higher order correlation among features is modelled using an autoencoder network. Stability is enhanced by combining a recent technique that uses a feature graph, and augmenting external unlabelled data for training the autoencoder network. Our experiments are conducted on a heart failure cohort from an Australian hospital. Stability was measured using Consistency index for feature subsets and signal-to-noise ratio for model parameters. Our methods demonstrated significant improvement in feature stability and model estimation stability when compared to baselines.

Original languageEnglish
Title of host publicationAdvanced Data Mining and Applications
Subtitle of host publication12th International Conference, ADMA 2016 Gold Coast, QLD, Australia, December 12–15, 2016 Proceedings
EditorsJianxin Li, Xue Li, Shuliang Wang, Jinyan Li, Quan Z. Sheng
Place of PublicationCham Switzerland
PublisherSpringer
Pages651-663
Number of pages13
ISBN (Electronic)9783319495866
ISBN (Print)9783319495859
DOIs
Publication statusPublished - 2016
Externally publishedYes
EventInternational Conference on Advanced Data Mining and Applications 2016 - Gold Coast, Australia
Duration: 12 Dec 201615 Dec 2016
Conference number: 12th
https://cs.adelaide.edu.au/~adma2016/
https://link.springer.com/book/10.1007/978-3-319-49586-6 (Proceedings)

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume10086
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Conference on Advanced Data Mining and Applications 2016
Abbreviated titleADMA 2016
Country/TerritoryAustralia
CityGold Coast
Period12/12/1615/12/16
Internet address

Cite this