Exploiting feature relationships towards stable feature selection

Iman Kamkar, Sunil Kumar Gupta, Dinh Phung, Svetha Venkatesh

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

8 Citations (Scopus)

Abstract

Feature selection is an important step in building predictive models for most real-world problems. One of the popular methods in feature selection is Lasso. However, it shows instability in selecting features when dealing with correlated features. In this work, we propose a new method that aims to increase the stability of Lasso by encouraging similarities between features based on their relatedness, which is captured via a feature covariance matrix. Besides modeling positive feature correlations, our method can also identify negative correlations between features. We propose a convex formulation for our model along with an alternating optimization algorithm that can learn the weights of the features as well as the relationship between them. Using both synthetic and real-world data, we show that the proposed method is more stable than Lasso and many state-of-the-art shrinkage and feature selection methods. Also, its predictive performance is comparable to other methods.

Original languageEnglish
Title of host publicationIEEE/ACM DSAA'2015 - Proceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics
Subtitle of host publication19-21 Oct 2015, Paris, France
EditorsEric Gaussier, Longbing Cao, Patrick Gallinari, James Kwok, Gabriella Pasi, Osmar Zaiane
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages10
ISBN (Electronic)9781467382731
ISBN (Print)781467382724
DOIs
Publication statusPublished - 2015
Externally publishedYes
EventIEEE International Conference on Data Science and Advanced Analytics 2015 - Paris, France
Duration: 19 Oct 201521 Oct 2015
Conference number: 2nd
http://dsaa2015.lip6.fr/
https://ieeexplore.ieee.org/xpl/conhome/7344768/proceeding (Proceedings)

Conference

ConferenceIEEE International Conference on Data Science and Advanced Analytics 2015
Abbreviated titleDSAA 2015
Country/TerritoryFrance
CityParis
Period19/10/1521/10/15
Internet address

Keywords

  • Correlated features
  • Lasso
  • Prediction
  • Stability

Cite this