Abstract
Logistic Regression (LR) is a workhorse of the statistics community and a state-of-the-art machine learning classifier. It learns a linear model from inputs to outputs trained by optimizing the Conditional Log-Likelihood (CLL) of the data. Recently, it has been shown that preconditioning LR using a Naive Bayes (NB) model speeds up LR learning many-fold. One can, however, train a linear model by optimizing the mean-square-error (MSE) instead of CLL. This leads to an Artificial Neural Network (ANN) with no hidden layer. In this work, we study the effect of NB preconditioning on such an ANN classifier. Optimizing MSE instead of CLL may lead to a lower bias classifier and hence result in better performance on big datasets. We show that this NB preconditioning can speed-up convergence significantly. We also show that optimizing a linear model with MSE leads to a lower bias classifier than optimizing with CLL. We also compare the performance to state-of-the-art classifier Random Forest.
| Original language | English |
|---|---|
| Title of host publication | Advances in Knowledge Discovery and Data Mining |
| Subtitle of host publication | 20th Pacific-Asia Conference, PAKDD 2016, Auckland, New Zealand, April 19-22, Proceedings, Part I |
| Editors | James Bailey, Latifur Khan, Takashi Washio, Gillian Dobbie, Joshua Zhexue Huang |
| Place of Publication | Switzerland |
| Publisher | Springer |
| Pages | 341-353 |
| Number of pages | 13 |
| Volume | 1 |
| ISBN (Electronic) | 9783319317533 |
| ISBN (Print) | 9783319317526 |
| DOIs | |
| Publication status | Published - 2016 |
| Event | Pacific-Asia Conference on Knowledge Discovery and Data Mining 2016 - Auckland, New Zealand Duration: 19 Apr 2016 → 22 Apr 2016 Conference number: 20th http://pakdd16.wordpress.fos.auckland.ac.nz/ |
Publication series
| Name | Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science) |
|---|---|
| Publisher | Springer |
| Volume | 9651 |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Conference
| Conference | Pacific-Asia Conference on Knowledge Discovery and Data Mining 2016 |
|---|---|
| Abbreviated title | PAKDD 2016 |
| Country | New Zealand |
| City | Auckland |
| Period | 19/04/16 → 22/04/16 |
| Internet address |
Keywords
- Artificial neural networks
- Conditional loglikelihood
- Logistic regression
- Mean-square-error
- Preconditioning
- WANBIA-C
Cite this
}
Preconditioning an artificial neural network using Naive Bayes. / Zaidi, Nayyar A.; Petitjean, François; Webb, Geoff I.
Advances in Knowledge Discovery and Data Mining: 20th Pacific-Asia Conference, PAKDD 2016, Auckland, New Zealand, April 19-22, Proceedings, Part I. ed. / James Bailey; Latifur Khan; Takashi Washio; Gillian Dobbie; Joshua Zhexue Huang. Vol. 1 Switzerland : Springer, 2016. p. 341-353 (Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science); Vol. 9651).Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review
TY - GEN
T1 - Preconditioning an artificial neural network using Naive Bayes
AU - Zaidi, Nayyar A.
AU - Petitjean, François
AU - Webb, Geoff I.
PY - 2016
Y1 - 2016
N2 - Logistic Regression (LR) is a workhorse of the statistics community and a state-of-the-art machine learning classifier. It learns a linear model from inputs to outputs trained by optimizing the Conditional Log-Likelihood (CLL) of the data. Recently, it has been shown that preconditioning LR using a Naive Bayes (NB) model speeds up LR learning many-fold. One can, however, train a linear model by optimizing the mean-square-error (MSE) instead of CLL. This leads to an Artificial Neural Network (ANN) with no hidden layer. In this work, we study the effect of NB preconditioning on such an ANN classifier. Optimizing MSE instead of CLL may lead to a lower bias classifier and hence result in better performance on big datasets. We show that this NB preconditioning can speed-up convergence significantly. We also show that optimizing a linear model with MSE leads to a lower bias classifier than optimizing with CLL. We also compare the performance to state-of-the-art classifier Random Forest.
AB - Logistic Regression (LR) is a workhorse of the statistics community and a state-of-the-art machine learning classifier. It learns a linear model from inputs to outputs trained by optimizing the Conditional Log-Likelihood (CLL) of the data. Recently, it has been shown that preconditioning LR using a Naive Bayes (NB) model speeds up LR learning many-fold. One can, however, train a linear model by optimizing the mean-square-error (MSE) instead of CLL. This leads to an Artificial Neural Network (ANN) with no hidden layer. In this work, we study the effect of NB preconditioning on such an ANN classifier. Optimizing MSE instead of CLL may lead to a lower bias classifier and hence result in better performance on big datasets. We show that this NB preconditioning can speed-up convergence significantly. We also show that optimizing a linear model with MSE leads to a lower bias classifier than optimizing with CLL. We also compare the performance to state-of-the-art classifier Random Forest.
KW - Artificial neural networks
KW - Conditional loglikelihood
KW - Logistic regression
KW - Mean-square-error
KW - Preconditioning
KW - WANBIA-C
UR - http://www.scopus.com/inward/record.url?scp=84964077894&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-31753-3_28
DO - 10.1007/978-3-319-31753-3_28
M3 - Conference Paper
AN - SCOPUS:84964077894
SN - 9783319317526
VL - 1
T3 - Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)
SP - 341
EP - 353
BT - Advances in Knowledge Discovery and Data Mining
A2 - Bailey, James
A2 - Khan, Latifur
A2 - Washio, Takashi
A2 - Dobbie, Gillian
A2 - Huang, Joshua Zhexue
PB - Springer
CY - Switzerland
ER -