Bayesian Multi-Hyperplane Machine for pattern recognition

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

Abstract

Current existing multi-hyperplane machine approach deals with high-dimensional and complex datasets by approximating the input data region using a parametric mixture of hyperplanes. Consequently, this approach requires an excessively time-consuming parameter search to find the set of optimal hyper-parameters. Another serious drawback of this approach is that it is often suboptimal since the optimal choice for the hyper-parameter is likely to lie outside the searching space due to the space discretization step required in grid search. To address these challenges, we propose in this paper BAyesian Multi-hyperplane Machine (BAMM). Our approach departs from a Bayesian perspective, and aims to construct an alternative probabilistic view in such a way that its maximum-a-posteriori (MAP) estimation reduces exactly to the original optimization problem of a multi-hyperplane machine. This view allows us to endow prior distributions over hyper-parameters and augment auxiliary variables to efficiently infer model parameters and hyper-parameters via Markov chain Monte Carlo (MCMC) method. We then employ a Stochastic Gradient Descent (SGD) framework to scale our model up with ever-growing large datasets. Extensive experiments demonstrate the capability of our proposed method in learning the optimal model without using any parameter tuning, and in achieving comparable accuracies compared with the state-of-art baselines; in the meantime our model can seamlessly handle with large-scale datasets.

Original languageEnglish
Title of host publication2018 24th International Conference on Pattern Recognition (ICPR)
Subtitle of host publicationAug. 20 2018 to Aug. 24 2018 Beijing, China
EditorsCheng-Lin Liu, Rama Chellappa, Matti Pietikäinen
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages609-614
Number of pages6
ISBN (Electronic)9781538637883, 9781538637876
ISBN (Print)97815386-37890
DOIs
Publication statusPublished - 2018
Externally publishedYes
EventInternational Conference on Pattern Recognition 2018 - Beijing, China
Duration: 20 Aug 201824 Aug 2018
Conference number: 24th
http://www.icpr2018.org/

Conference

ConferenceInternational Conference on Pattern Recognition 2018
Abbreviated titleICPR 2018
CountryChina
CityBeijing
Period20/08/1824/08/18
Internet address

Cite this

Nguyen, K., Le, T., Nguyen, T. D., & Phung, D. (2018). Bayesian Multi-Hyperplane Machine for pattern recognition. In C-L. Liu, R. Chellappa, & M. Pietikäinen (Eds.), 2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China (pp. 609-614). [8545139] Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICPR.2018.8545139
Nguyen, Khanh ; Le, Trung ; Nguyen, Tu Dinh ; Phung, Dinh. / Bayesian Multi-Hyperplane Machine for pattern recognition. 2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China. editor / Cheng-Lin Liu ; Rama Chellappa ; Matti Pietikäinen. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. pp. 609-614
@inproceedings{4084c808c21e40b7a43c2ed7df0ea634,
title = "Bayesian Multi-Hyperplane Machine for pattern recognition",
abstract = "Current existing multi-hyperplane machine approach deals with high-dimensional and complex datasets by approximating the input data region using a parametric mixture of hyperplanes. Consequently, this approach requires an excessively time-consuming parameter search to find the set of optimal hyper-parameters. Another serious drawback of this approach is that it is often suboptimal since the optimal choice for the hyper-parameter is likely to lie outside the searching space due to the space discretization step required in grid search. To address these challenges, we propose in this paper BAyesian Multi-hyperplane Machine (BAMM). Our approach departs from a Bayesian perspective, and aims to construct an alternative probabilistic view in such a way that its maximum-a-posteriori (MAP) estimation reduces exactly to the original optimization problem of a multi-hyperplane machine. This view allows us to endow prior distributions over hyper-parameters and augment auxiliary variables to efficiently infer model parameters and hyper-parameters via Markov chain Monte Carlo (MCMC) method. We then employ a Stochastic Gradient Descent (SGD) framework to scale our model up with ever-growing large datasets. Extensive experiments demonstrate the capability of our proposed method in learning the optimal model without using any parameter tuning, and in achieving comparable accuracies compared with the state-of-art baselines; in the meantime our model can seamlessly handle with large-scale datasets.",
author = "Khanh Nguyen and Trung Le and Nguyen, {Tu Dinh} and Dinh Phung",
year = "2018",
doi = "10.1109/ICPR.2018.8545139",
language = "English",
isbn = "97815386-37890",
pages = "609--614",
editor = "Cheng-Lin Liu and Chellappa, {Rama } and Pietik{\"a}inen, {Matti }",
booktitle = "2018 24th International Conference on Pattern Recognition (ICPR)",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
address = "United States of America",

}

Nguyen, K, Le, T, Nguyen, TD & Phung, D 2018, Bayesian Multi-Hyperplane Machine for pattern recognition. in C-L Liu, R Chellappa & M Pietikäinen (eds), 2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China., 8545139, IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 609-614, International Conference on Pattern Recognition 2018, Beijing, China, 20/08/18. https://doi.org/10.1109/ICPR.2018.8545139

Bayesian Multi-Hyperplane Machine for pattern recognition. / Nguyen, Khanh; Le, Trung; Nguyen, Tu Dinh; Phung, Dinh.

2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China. ed. / Cheng-Lin Liu; Rama Chellappa; Matti Pietikäinen. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2018. p. 609-614 8545139.

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

TY - GEN

T1 - Bayesian Multi-Hyperplane Machine for pattern recognition

AU - Nguyen, Khanh

AU - Le, Trung

AU - Nguyen, Tu Dinh

AU - Phung, Dinh

PY - 2018

Y1 - 2018

N2 - Current existing multi-hyperplane machine approach deals with high-dimensional and complex datasets by approximating the input data region using a parametric mixture of hyperplanes. Consequently, this approach requires an excessively time-consuming parameter search to find the set of optimal hyper-parameters. Another serious drawback of this approach is that it is often suboptimal since the optimal choice for the hyper-parameter is likely to lie outside the searching space due to the space discretization step required in grid search. To address these challenges, we propose in this paper BAyesian Multi-hyperplane Machine (BAMM). Our approach departs from a Bayesian perspective, and aims to construct an alternative probabilistic view in such a way that its maximum-a-posteriori (MAP) estimation reduces exactly to the original optimization problem of a multi-hyperplane machine. This view allows us to endow prior distributions over hyper-parameters and augment auxiliary variables to efficiently infer model parameters and hyper-parameters via Markov chain Monte Carlo (MCMC) method. We then employ a Stochastic Gradient Descent (SGD) framework to scale our model up with ever-growing large datasets. Extensive experiments demonstrate the capability of our proposed method in learning the optimal model without using any parameter tuning, and in achieving comparable accuracies compared with the state-of-art baselines; in the meantime our model can seamlessly handle with large-scale datasets.

AB - Current existing multi-hyperplane machine approach deals with high-dimensional and complex datasets by approximating the input data region using a parametric mixture of hyperplanes. Consequently, this approach requires an excessively time-consuming parameter search to find the set of optimal hyper-parameters. Another serious drawback of this approach is that it is often suboptimal since the optimal choice for the hyper-parameter is likely to lie outside the searching space due to the space discretization step required in grid search. To address these challenges, we propose in this paper BAyesian Multi-hyperplane Machine (BAMM). Our approach departs from a Bayesian perspective, and aims to construct an alternative probabilistic view in such a way that its maximum-a-posteriori (MAP) estimation reduces exactly to the original optimization problem of a multi-hyperplane machine. This view allows us to endow prior distributions over hyper-parameters and augment auxiliary variables to efficiently infer model parameters and hyper-parameters via Markov chain Monte Carlo (MCMC) method. We then employ a Stochastic Gradient Descent (SGD) framework to scale our model up with ever-growing large datasets. Extensive experiments demonstrate the capability of our proposed method in learning the optimal model without using any parameter tuning, and in achieving comparable accuracies compared with the state-of-art baselines; in the meantime our model can seamlessly handle with large-scale datasets.

UR - http://www.scopus.com/inward/record.url?scp=85059740253&partnerID=8YFLogxK

U2 - 10.1109/ICPR.2018.8545139

DO - 10.1109/ICPR.2018.8545139

M3 - Conference Paper

SN - 97815386-37890

SP - 609

EP - 614

BT - 2018 24th International Conference on Pattern Recognition (ICPR)

A2 - Liu, Cheng-Lin

A2 - Chellappa, Rama

A2 - Pietikäinen, Matti

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

ER -

Nguyen K, Le T, Nguyen TD, Phung D. Bayesian Multi-Hyperplane Machine for pattern recognition. In Liu C-L, Chellappa R, Pietikäinen M, editors, 2018 24th International Conference on Pattern Recognition (ICPR): Aug. 20 2018 to Aug. 24 2018 Beijing, China. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2018. p. 609-614. 8545139 https://doi.org/10.1109/ICPR.2018.8545139