Abstract
Stein variational gradient descent (SVGD) was recently proposed as a general purpose nonparametric variational inference algorithm [Liu & Wang, NIPS 2016]: it minimizes the Kullback-Leibler divergence between the target distribution and its approximation by implementing a form of functional gradient descent on a reproducing kernel Hilbert space. In this paper, we accelerate and generalize the SVGD algorithm by including second-order information, thereby approximating a Newton-like iteration in function space. We also show how second-order information can lead to more effective choices of kernel. We observe significant computational gains over the original SVGD algorithm in multiple test cases.
Original language | English |
---|---|
Title of host publication | Proceedings of Advances in Neural Information Processing Systems |
Subtitle of host publication | 32nd Conference on Neural Information Processing Systems, NeurIPS 2018; Montreal; Canada; 2 December 2018 through 8 December 2018 |
Editors | S Bengio, H Wallach, H Larochelle, K Grauman, N Cesa-Bianchi, R Garnett |
Place of Publication | San Diego CA USA |
Number of pages | 11 |
Volume | 2018-December |
Publication status | Published - 1 Jan 2018 |
Event | Advances in Neural Information Processing Systems 2018 - Montreal Convention Center (Palais des Congrès de Montréal), Montreal , Canada Duration: 2 Dec 2018 → 8 Dec 2018 Conference number: 31st https://papers.nips.cc/book/advances-in-neural-information-processing-systems-31-2018 (Proceedings) |
Publication series
Name | Advances in Neural Information Processing Systems |
---|---|
ISSN (Print) | 1049-5258 |
Conference
Conference | Advances in Neural Information Processing Systems 2018 |
---|---|
Abbreviated title | NIPS 2018 |
Country | Canada |
City | Montreal |
Period | 2/12/18 → 8/12/18 |
Other | The Annual Conference on Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting. |
Internet address |