Abstract
Feature selection is an important step in building predictive models for most real-world problems. One of the popular methods in feature selection is Lasso. However, it shows instability in selecting features when dealing with correlated features. In this work, we propose a new method that aims to increase the stability of Lasso by encouraging similarities between features based on their relatedness, which is captured via a feature covariance matrix. Besides modeling positive feature correlations, our method can also identify negative correlations between features. We propose a convex formulation for our model along with an alternating optimization algorithm that can learn the weights of the features as well as the relationship between them. Using both synthetic and real-world data, we show that the proposed method is more stable than Lasso and many state-of-the-art shrinkage and feature selection methods. Also, its predictive performance is comparable to other methods.
Original language | English |
---|---|
Title of host publication | IEEE/ACM DSAA'2015 - Proceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics |
Subtitle of host publication | 19-21 Oct 2015, Paris, France |
Editors | Eric Gaussier, Longbing Cao, Patrick Gallinari, James Kwok, Gabriella Pasi, Osmar Zaiane |
Place of Publication | Piscataway NJ USA |
Publisher | IEEE, Institute of Electrical and Electronics Engineers |
Number of pages | 10 |
ISBN (Electronic) | 9781467382731 |
ISBN (Print) | 781467382724 |
DOIs | |
Publication status | Published - 2015 |
Externally published | Yes |
Event | IEEE International Conference on Data Science and Advanced Analytics 2015 - Paris, France Duration: 19 Oct 2015 → 21 Oct 2015 Conference number: 2nd http://dsaa2015.lip6.fr/ https://ieeexplore.ieee.org/xpl/conhome/7344768/proceeding (Proceedings) |
Conference
Conference | IEEE International Conference on Data Science and Advanced Analytics 2015 |
---|---|
Abbreviated title | DSAA 2015 |
Country/Territory | France |
City | Paris |
Period | 19/10/15 → 21/10/15 |
Internet address |
Keywords
- Correlated features
- Lasso
- Prediction
- Stability