Abstract
For their strong generalization ability, especially on limited data to effectively generalize on unseen data. However, most kernel methods, including the state-of-the-art LIBSVM, are vulnerable to the curse of kernelization, making them infeasible to apply to large-scale datasets. This issue is exacerbated when kernel methods are used in conjunction with a grid search to tune their kernel parameters and hyperparameters which brings in the question of model robustness when applied to real datasets. In this paper, we propose a robust Bayesian Kernel Machine (BKM) - a Bayesian kernel machine that exploits the strengths of both the Bayesian modelling and kernel methods. A key challenge for such a formulation is the need for an efficient learning algorithm. To this end, we successfully extended the recent Stein variational theory for Bayesian inference for our proposed model, resulting in fast and efficient learning and prediction algorithms. Importantly our proposed BKM is resilient to the curse of kernelization, hence making it applicable to large-scale datasets and robust to parameter tuning, avoiding the associated expense and potential pitfalls with current practice of parameter tuning. Our extensive experimental results on 12 benchmark datasets show that our BKM without tuning any parameter can achieve comparable predictive performance with the state-of-the-art LIBSVM and significantly outperforms other baselines, while obtaining significantly speedup in terms of the total training time compared with its rivals.
Original language | English |
---|---|
Title of host publication | KDD'18 |
Subtitle of host publication | Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining |
Editors | Chih-Jen Lin, Hui Xiong |
Place of Publication | New York NY USA |
Publisher | Association for Computing Machinery (ACM) |
Pages | 2003-2011 |
Number of pages | 9 |
ISBN (Print) | 9781450355520 |
DOIs | |
Publication status | Published - 19 Jul 2018 |
Event | ACM International Conference on Knowledge Discovery and Data Mining 2018 - London, United Kingdom Duration: 19 Aug 2018 → 23 Aug 2018 Conference number: 24th http://www.kdd.org/kdd2018/ (Conference website) https://dl.acm.org/doi/proceedings/10.1145/3219819 |
Conference
Conference | ACM International Conference on Knowledge Discovery and Data Mining 2018 |
---|---|
Abbreviated title | KDD 2018 |
Country/Territory | United Kingdom |
City | London |
Period | 19/08/18 → 23/08/18 |
Internet address |
|
Keywords
- Bayesian inference
- Big data
- Kernel methods
- Multiclass supervised learning
- Random feature
- Stein divergence
- Variational method