Review of modern logistic regression methods with application to small and medium sample size problems

Enes Makalic; Daniel Francis Schmidt

doi:10.1007/978-3-642-17432-2_22

Review of modern logistic regression methods with application to small and medium sample size problems

Enes Makalic, Daniel Francis Schmidt

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

17 Citations (Scopus)

Abstract

Logistic regression is one of the most widely applied machine learning tools in binary classification problems. Traditionally, inference of logistic models has focused on stepwise regression procedures which determine the predictor variables to be included in the model. Techniques that modify the log-likelihood by adding a continuous penalty function of the parameters have recently been used when inferring logistic models with a large number of predictor variables. This paper compares and contrasts three popular penalized logistic regression methods: ridge regression, the Least Absolute Shrinkage and Selection Operator (LASSO) and the elastic net. The methods are compared in terms of prediction accuracy using simulated data as well as real data sets.

Original language	English
Title of host publication	AI 2010
Subtitle of host publication	Advances in Artificial Intelligence - 23rd Australasian Joint Conference, Proceedings
Publisher	Springer
Pages	213-222
Number of pages	10
ISBN (Print)	3642174310, 9783642174315
DOIs	https://doi.org/10.1007/978-3-642-17432-2_22
Publication status	Published - 2010
Externally published	Yes
Event	Australasian Joint Conference on Artificial Intelligence 2010 - Adelaide, Australia Duration: 7 Dec 2010 → 10 Dec 2010 Conference number: 23rd https://link.springer.com/book/10.1007/978-3-642-17432-2 (Proceedings)

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	6464
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Australasian Joint Conference on Artificial Intelligence 2010
Abbreviated title	AI 2010
Country/Territory	Australia
City	Adelaide
Period	7/12/10 → 10/12/10
Internet address	https://link.springer.com/book/10.1007/978-3-642-17432-2 (Proceedings)

Keywords

Elastic Net
LASSO
Logistic regression
Ridge regression
Variable Selection

Access to Document

10.1007/978-3-642-17432-2_22

Cite this

@inproceedings{521db7792e1c4e42b581df2b3e102505,

title = "Review of modern logistic regression methods with application to small and medium sample size problems",

abstract = "Logistic regression is one of the most widely applied machine learning tools in binary classification problems. Traditionally, inference of logistic models has focused on stepwise regression procedures which determine the predictor variables to be included in the model. Techniques that modify the log-likelihood by adding a continuous penalty function of the parameters have recently been used when inferring logistic models with a large number of predictor variables. This paper compares and contrasts three popular penalized logistic regression methods: ridge regression, the Least Absolute Shrinkage and Selection Operator (LASSO) and the elastic net. The methods are compared in terms of prediction accuracy using simulated data as well as real data sets.",

keywords = "Elastic Net, LASSO, Logistic regression, Ridge regression, Variable Selection",

author = "Enes Makalic and Schmidt, {Daniel Francis}",

year = "2010",

doi = "10.1007/978-3-642-17432-2_22",

language = "English",

isbn = "3642174310",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "213--222",

booktitle = "AI 2010",

note = "Australasian Joint Conference on Artificial Intelligence 2010, AI 2010 ; Conference date: 07-12-2010 Through 10-12-2010",

url = "https://link.springer.com/book/10.1007/978-3-642-17432-2",

}

Makalic, E & Schmidt, DF 2010, Review of modern logistic regression methods with application to small and medium sample size problems. in AI 2010: Advances in Artificial Intelligence - 23rd Australasian Joint Conference, Proceedings. Lecture Notes in Computer Science, vol. 6464 , Springer, pp. 213-222, Australasian Joint Conference on Artificial Intelligence 2010, Adelaide, South Australia, Australia, 7/12/10. https://doi.org/10.1007/978-3-642-17432-2_22

Review of modern logistic regression methods with application to small and medium sample size problems. / Makalic, Enes ; Schmidt, Daniel Francis.
AI 2010: Advances in Artificial Intelligence - 23rd Australasian Joint Conference, Proceedings. Springer, 2010. p. 213-222 (Lecture Notes in Computer Science; Vol. 6464 ).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Review of modern logistic regression methods with application to small and medium sample size problems

AU - Makalic, Enes

AU - Schmidt, Daniel Francis

N1 - Conference code: 23rd

PY - 2010

Y1 - 2010

N2 - Logistic regression is one of the most widely applied machine learning tools in binary classification problems. Traditionally, inference of logistic models has focused on stepwise regression procedures which determine the predictor variables to be included in the model. Techniques that modify the log-likelihood by adding a continuous penalty function of the parameters have recently been used when inferring logistic models with a large number of predictor variables. This paper compares and contrasts three popular penalized logistic regression methods: ridge regression, the Least Absolute Shrinkage and Selection Operator (LASSO) and the elastic net. The methods are compared in terms of prediction accuracy using simulated data as well as real data sets.

AB - Logistic regression is one of the most widely applied machine learning tools in binary classification problems. Traditionally, inference of logistic models has focused on stepwise regression procedures which determine the predictor variables to be included in the model. Techniques that modify the log-likelihood by adding a continuous penalty function of the parameters have recently been used when inferring logistic models with a large number of predictor variables. This paper compares and contrasts three popular penalized logistic regression methods: ridge regression, the Least Absolute Shrinkage and Selection Operator (LASSO) and the elastic net. The methods are compared in terms of prediction accuracy using simulated data as well as real data sets.

KW - Elastic Net

KW - LASSO

KW - Logistic regression

KW - Ridge regression

KW - Variable Selection

UR - http://www.scopus.com/inward/record.url?scp=78650795388&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-17432-2_22

DO - 10.1007/978-3-642-17432-2_22

M3 - Conference Paper

AN - SCOPUS:78650795388

SN - 3642174310

SN - 9783642174315

T3 - Lecture Notes in Computer Science

SP - 213

EP - 222

BT - AI 2010

PB - Springer

T2 - Australasian Joint Conference on Artificial Intelligence 2010

Y2 - 7 December 2010 through 10 December 2010

ER -

Review of modern logistic regression methods with application to small and medium sample size problems

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Cite this