Inter and intra topic structure learning with word embeddings

He Zhao, Lan Du, Wray Buntine, Mingyuan Zhou

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

One important task of topic modeling for text analysis is interpretability. By discovering structured topics one is able to yield improved interpretability as well as modeling accuracy. In this paper, we propose a novel topic model with a deep structure that explores both inter-topic and intra-topic structures informed by word embeddings. Specifically, our model discovers inter topic structures in the form of topic hierarchies and discovers intra topic structures in the form of sub-topics, each of which is informed by word embeddings and captures a fine-grained thematic aspect of a normal topic. Extensive experiments demonstrate that our model achieves the state-of-the-art performance in terms of perplexity, document classification, and topic quality. Moreover, with topic hierarchies and sub-topics, the topics discovered in our model are more interpretable, providing an illuminating means to understand text data.
Original languageEnglish
Title of host publicationProceedings of Machine Learning Research
Subtitle of host publicationInternational Conference on Machine Learning, 10-15 July 2018, Stockholmsmässan, Stockholm Sweden
EditorsJennifer Dy, Andreas Krause
Place of PublicationStockholmsmässan Stockholm Sweden
PublisherProceedings of Machine Learning Research (PMLR)
Number of pages10
Volume80
ISBN (Electronic)9781510867963
Publication statusPublished - 2018
EventInternational Conference on Machine Learning 2018 - Stockholm, Sweden
Duration: 10 Jul 201815 Jul 2018

Publication series

NameProceedings of Machine Learning Research
ISSN (Print)1938-7228

Conference

ConferenceInternational Conference on Machine Learning 2018
CountrySweden
CityStockholm
Period10/07/1815/07/18

Cite this

Zhao, H., Du, L., Buntine, W., & Zhou, M. (2018). Inter and intra topic structure learning with word embeddings. In J. Dy, & A. Krause (Eds.), Proceedings of Machine Learning Research: International Conference on Machine Learning, 10-15 July 2018, Stockholmsmässan, Stockholm Sweden (Vol. 80). (Proceedings of Machine Learning Research). Stockholmsmässan Stockholm Sweden: Proceedings of Machine Learning Research (PMLR).
Zhao, He ; Du, Lan ; Buntine, Wray ; Zhou, Mingyuan. / Inter and intra topic structure learning with word embeddings. Proceedings of Machine Learning Research: International Conference on Machine Learning, 10-15 July 2018, Stockholmsmässan, Stockholm Sweden. editor / Jennifer Dy ; Andreas Krause. Vol. 80 Stockholmsmässan Stockholm Sweden : Proceedings of Machine Learning Research (PMLR), 2018. (Proceedings of Machine Learning Research).
@inproceedings{5162e9c524674dc3aed44b6b3a0d850e,
title = "Inter and intra topic structure learning with word embeddings",
abstract = "One important task of topic modeling for text analysis is interpretability. By discovering structured topics one is able to yield improved interpretability as well as modeling accuracy. In this paper, we propose a novel topic model with a deep structure that explores both inter-topic and intra-topic structures informed by word embeddings. Specifically, our model discovers inter topic structures in the form of topic hierarchies and discovers intra topic structures in the form of sub-topics, each of which is informed by word embeddings and captures a fine-grained thematic aspect of a normal topic. Extensive experiments demonstrate that our model achieves the state-of-the-art performance in terms of perplexity, document classification, and topic quality. Moreover, with topic hierarchies and sub-topics, the topics discovered in our model are more interpretable, providing an illuminating means to understand text data.",
author = "He Zhao and Lan Du and Wray Buntine and Mingyuan Zhou",
year = "2018",
language = "English",
volume = "80",
series = "Proceedings of Machine Learning Research",
publisher = "Proceedings of Machine Learning Research (PMLR)",
editor = "Dy, {Jennifer } and Krause, {Andreas }",
booktitle = "Proceedings of Machine Learning Research",

}

Zhao, H, Du, L, Buntine, W & Zhou, M 2018, Inter and intra topic structure learning with word embeddings. in J Dy & A Krause (eds), Proceedings of Machine Learning Research: International Conference on Machine Learning, 10-15 July 2018, Stockholmsmässan, Stockholm Sweden. vol. 80, Proceedings of Machine Learning Research, Proceedings of Machine Learning Research (PMLR), Stockholmsmässan Stockholm Sweden, International Conference on Machine Learning 2018, Stockholm, Sweden, 10/07/18.

Inter and intra topic structure learning with word embeddings. / Zhao, He; Du, Lan; Buntine, Wray; Zhou, Mingyuan.

Proceedings of Machine Learning Research: International Conference on Machine Learning, 10-15 July 2018, Stockholmsmässan, Stockholm Sweden. ed. / Jennifer Dy; Andreas Krause. Vol. 80 Stockholmsmässan Stockholm Sweden : Proceedings of Machine Learning Research (PMLR), 2018. (Proceedings of Machine Learning Research).

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - Inter and intra topic structure learning with word embeddings

AU - Zhao, He

AU - Du, Lan

AU - Buntine, Wray

AU - Zhou, Mingyuan

PY - 2018

Y1 - 2018

N2 - One important task of topic modeling for text analysis is interpretability. By discovering structured topics one is able to yield improved interpretability as well as modeling accuracy. In this paper, we propose a novel topic model with a deep structure that explores both inter-topic and intra-topic structures informed by word embeddings. Specifically, our model discovers inter topic structures in the form of topic hierarchies and discovers intra topic structures in the form of sub-topics, each of which is informed by word embeddings and captures a fine-grained thematic aspect of a normal topic. Extensive experiments demonstrate that our model achieves the state-of-the-art performance in terms of perplexity, document classification, and topic quality. Moreover, with topic hierarchies and sub-topics, the topics discovered in our model are more interpretable, providing an illuminating means to understand text data.

AB - One important task of topic modeling for text analysis is interpretability. By discovering structured topics one is able to yield improved interpretability as well as modeling accuracy. In this paper, we propose a novel topic model with a deep structure that explores both inter-topic and intra-topic structures informed by word embeddings. Specifically, our model discovers inter topic structures in the form of topic hierarchies and discovers intra topic structures in the form of sub-topics, each of which is informed by word embeddings and captures a fine-grained thematic aspect of a normal topic. Extensive experiments demonstrate that our model achieves the state-of-the-art performance in terms of perplexity, document classification, and topic quality. Moreover, with topic hierarchies and sub-topics, the topics discovered in our model are more interpretable, providing an illuminating means to understand text data.

UR - http://www.scopus.com/inward/record.url?scp=85057245542&partnerID=8YFLogxK

M3 - Conference Paper

VL - 80

T3 - Proceedings of Machine Learning Research

BT - Proceedings of Machine Learning Research

A2 - Dy, Jennifer

A2 - Krause, Andreas

PB - Proceedings of Machine Learning Research (PMLR)

CY - Stockholmsmässan Stockholm Sweden

ER -

Zhao H, Du L, Buntine W, Zhou M. Inter and intra topic structure learning with word embeddings. In Dy J, Krause A, editors, Proceedings of Machine Learning Research: International Conference on Machine Learning, 10-15 July 2018, Stockholmsmässan, Stockholm Sweden. Vol. 80. Stockholmsmässan Stockholm Sweden: Proceedings of Machine Learning Research (PMLR). 2018. (Proceedings of Machine Learning Research).