Robust Bayesian kernel machine via Stein variational gradient descent for big data

Khanh Nguyen, Trung Le, Tu Dinh Nguyen, Dinh Phung, Geoffrey I. Webb

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

2 Citations (Scopus)

Abstract

For their strong generalization ability, especially on limited data to effectively generalize on unseen data. However, most kernel methods, including the state-of-the-art LIBSVM, are vulnerable to the curse of kernelization, making them infeasible to apply to large-scale datasets. This issue is exacerbated when kernel methods are used in conjunction with a grid search to tune their kernel parameters and hyperparameters which brings in the question of model robustness when applied to real datasets. In this paper, we propose a robust Bayesian Kernel Machine (BKM) - a Bayesian kernel machine that exploits the strengths of both the Bayesian modelling and kernel methods. A key challenge for such a formulation is the need for an efficient learning algorithm. To this end, we successfully extended the recent Stein variational theory for Bayesian inference for our proposed model, resulting in fast and efficient learning and prediction algorithms. Importantly our proposed BKM is resilient to the curse of kernelization, hence making it applicable to large-scale datasets and robust to parameter tuning, avoiding the associated expense and potential pitfalls with current practice of parameter tuning. Our extensive experimental results on 12 benchmark datasets show that our BKM without tuning any parameter can achieve comparable predictive performance with the state-of-the-art LIBSVM and significantly outperforms other baselines, while obtaining significantly speedup in terms of the total training time compared with its rivals.

Original languageEnglish
Title of host publicationKDD'18
Subtitle of host publicationProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
EditorsChih-Jen Lin, Hui Xiong
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages2003-2011
Number of pages9
ISBN (Print)9781450355520
DOIs
Publication statusPublished - 19 Jul 2018
EventACM International Conference on Knowledge Discovery and Data Mining 2018 - London, United Kingdom
Duration: 19 Aug 201823 Aug 2018
Conference number: 24th
http://www.kdd.org/kdd2018/ (Conference website)

Conference

ConferenceACM International Conference on Knowledge Discovery and Data Mining 2018
Abbreviated titleSIGKDD 2018
CountryUnited Kingdom
CityLondon
Period19/08/1823/08/18
Internet address

Keywords

  • Bayesian inference
  • Big data
  • Kernel methods
  • Multiclass supervised learning
  • Random feature
  • Stein divergence
  • Variational method

Cite this

Nguyen, K., Le, T., Nguyen, T. D., Phung, D., & Webb, G. I. (2018). Robust Bayesian kernel machine via Stein variational gradient descent for big data. In C-J. Lin, & H. Xiong (Eds.), KDD'18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 2003-2011). Association for Computing Machinery (ACM). https://doi.org/10.1145/3219819.3220015