Doubly robust Bayesian inference for non-stationary streaming data with β-divergences

Jeremias Knoblauch; Jack Jewson; Theodoros Damoulas

Doubly robust Bayesian inference for non-stationary streaming data with β-divergences

Jeremias Knoblauch, Jack Jewson, Theodoros Damoulas

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other › peer-review

Abstract

We present the first robust Bayesian Online Changepoint Detection algorithm through General Bayesian Inference (GBI) with β-divergences. The resulting inference procedure is doubly robust for both the parameter and the changepoint (CP) posterior, with linear time and constant space complexity. We provide a construction for exponential models and demonstrate it on the Bayesian Linear Regression model. In so doing, we make two additional contributions: Firstly, we make GBI scalable using Structural Variational approximations that are exact as β → 0. Secondly, we give a principled way of choosing the divergence parameter β by minimizing expected predictive loss on-line. Reducing False Discovery Rates of CPS from over 90% to 0% on real world data, this offers the state of the art.

Original language	English
Title of host publication	Proceedings of Advances in Neural Information Processing Systems 2018
Pages	64-75
Number of pages	12
Publication status	Published - 2018
Externally published	Yes
Event	Advances in Neural Information Processing Systems 2018 - Montreal Convention Center (Palais des Congrès de Montréal), Montreal , Canada Duration: 2 Dec 2018 → 8 Dec 2018 Conference number: 31st https://papers.nips.cc/book/advances-in-neural-information-processing-systems-31-2018 (Proceedings)

Conference

Conference	Advances in Neural Information Processing Systems 2018
Abbreviated title	NIPS 2018
Country/Territory	Canada
City	Montreal
Period	2/12/18 → 8/12/18
Other	The Annual Conference on Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Internet address	https://papers.nips.cc/book/advances-in-neural-information-processing-systems-31-2018 (Proceedings)

Access to Document

https://papers.nips.cc/paper_files/paper/2018/hash/a3f390d88e4c41f2747bfa2f1b5f87db-Abstract.html

Cite this

@inproceedings{330da421a90448e296a600740ce0c372,

title = "Doubly robust Bayesian inference for non-stationary streaming data with β-divergences",

abstract = "We present the first robust Bayesian Online Changepoint Detection algorithm through General Bayesian Inference (GBI) with β-divergences. The resulting inference procedure is doubly robust for both the parameter and the changepoint (CP) posterior, with linear time and constant space complexity. We provide a construction for exponential models and demonstrate it on the Bayesian Linear Regression model. In so doing, we make two additional contributions: Firstly, we make GBI scalable using Structural Variational approximations that are exact as β → 0. Secondly, we give a principled way of choosing the divergence parameter β by minimizing expected predictive loss on-line. Reducing False Discovery Rates of CPS from over 90\% to 0\% on real world data, this offers the state of the art.",

author = "Jeremias Knoblauch and Jack Jewson and Theodoros Damoulas",

note = "Funding Information: We would like to cordially thank both Jim Smith and Chris Holmes for fruitful discussions and help with some of the theoretical results. JK and JJ are funded by EPSRC grant EP/L016710/1 as part of the Oxford-Warwick Statistics Programme (OXWASP). TD is funded by the Lloyds Register Foundation programme on Data Centric Engineering through the London Air Quality project. This work was supported by The Alan Turing Institute for Data Science and AI under EPSRC grant EP/N510129/1. In collaboration with the Greater London Authority. Publisher Copyright: {\textcopyright} 2018 Curran Associates Inc..All rights reserved.; Advances in Neural Information Processing Systems 2018, NIPS 2018 ; Conference date: 02-12-2018 Through 08-12-2018",

year = "2018",

language = "English",

pages = "64--75",

booktitle = "Proceedings of Advances in Neural Information Processing Systems 2018",

url = "https://papers.nips.cc/book/advances-in-neural-information-processing-systems-31-2018",

}

Knoblauch, J, Jewson, J & Damoulas, T 2018, Doubly robust Bayesian inference for non-stationary streaming data with β-divergences. in Proceedings of Advances in Neural Information Processing Systems 2018. pp. 64-75, Advances in Neural Information Processing Systems 2018, Montreal , Quebec, Canada, 2/12/18. <https://papers.nips.cc/paper_files/paper/2018/hash/a3f390d88e4c41f2747bfa2f1b5f87db-Abstract.html>

TY - GEN

T1 - Doubly robust Bayesian inference for non-stationary streaming data with β-divergences

AU - Knoblauch, Jeremias

AU - Jewson, Jack

AU - Damoulas, Theodoros

N1 - Conference code: 31st

PY - 2018

Y1 - 2018

N2 - We present the first robust Bayesian Online Changepoint Detection algorithm through General Bayesian Inference (GBI) with β-divergences. The resulting inference procedure is doubly robust for both the parameter and the changepoint (CP) posterior, with linear time and constant space complexity. We provide a construction for exponential models and demonstrate it on the Bayesian Linear Regression model. In so doing, we make two additional contributions: Firstly, we make GBI scalable using Structural Variational approximations that are exact as β → 0. Secondly, we give a principled way of choosing the divergence parameter β by minimizing expected predictive loss on-line. Reducing False Discovery Rates of CPS from over 90% to 0% on real world data, this offers the state of the art.

AB - We present the first robust Bayesian Online Changepoint Detection algorithm through General Bayesian Inference (GBI) with β-divergences. The resulting inference procedure is doubly robust for both the parameter and the changepoint (CP) posterior, with linear time and constant space complexity. We provide a construction for exponential models and demonstrate it on the Bayesian Linear Regression model. In so doing, we make two additional contributions: Firstly, we make GBI scalable using Structural Variational approximations that are exact as β → 0. Secondly, we give a principled way of choosing the divergence parameter β by minimizing expected predictive loss on-line. Reducing False Discovery Rates of CPS from over 90% to 0% on real world data, this offers the state of the art.

UR - https://www.scopus.com/pages/publications/85064820879

M3 - Conference Paper

AN - SCOPUS:85064820879

SP - 64

EP - 75

BT - Proceedings of Advances in Neural Information Processing Systems 2018

T2 - Advances in Neural Information Processing Systems 2018

Y2 - 2 December 2018 through 8 December 2018

ER -

Doubly robust Bayesian inference for non-stationary streaming data with β-divergences

Abstract

Conference

Access to Document

Other files and links

Cite this