Supervised restricted Boltzmann machines

Tu Dinh Nguyen, Dinh Phung, Viet Huynh, Trung Le

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

4 Citations (Scopus)


We propose in this paper the supervised re- stricted Boltzmann machine (sRBM), a unified framework which combines the versatility of RBM to simultaneously learn the data representation and to perform supervised learning (i.e., a nonlinear classifier or a nonlinear regressor). Unlike the current state-of-the-art classification formulation proposed for RBM in (Larochelle et al., 2012), our model is a hybrid probabilistic graphical model consisting of a distinguished genera- tive component for data representation and a dis- criminative component for prediction. While the work of (Larochelle et al., 2012) typically incurs no extra difficulty in inference compared with a standard RBM, our discriminative component, modeled as a directed graphical model, renders MCMC-based inference (e.g., Gibbs sampler) very slow and unpractical for use. To this end, we further develop scalable variational inference for the proposed sRBM for both classification and regression cases. Extensive experiments on realworld datasets show that our sRBM achieves better predictive performance than baseline methods. At the same time, our proposed framework yields learned representations which are more discriminative, hence interpretable, than those of its counterparts. Besides, our method is probabilistic and capable of generating meaningful data conditioning on specific classes - a topic which is of current great interest in deep learning aiming at data generation.

Original languageEnglish
Title of host publicationConference on Uncertainty in Artifical Intelligence, UAI 2017
Subtitle of host publicationSydney, Australia August 11-15, 2017
EditorsGal Elidan , Kristian Kersting
Place of PublicationUSA
PublisherAUAI Press
Number of pages10
Publication statusPublished - 2017
Externally publishedYes
EventConference in Uncertainty in Artificial Intelligence 2017 - Sydney, Australia
Duration: 11 Aug 201715 Aug 2017
Conference number: 33rd


ConferenceConference in Uncertainty in Artificial Intelligence 2017
Abbreviated titleUAI 2017

Cite this