A Word Embeddings Informed Focused Topic Model

He Zhao, Lan Du, Wray Buntine

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch


    In natural language processing and related fields, it has been shown that the word embeddings can successfully capture both the semantic and syntactic features of words. They can serve as complementary information to topics models, especially for the cases where word co-occurrence data is insufficient, such as with short texts. In this paper, we propose a focused topic model where how a topic focuses on words is informed by word embeddings. Our models is able to discover more informed and focused topics with more representative words, leading to better modelling accuracy and topic quality. With the data argumentation technique, we can derive an efficient Gibbs sampling algorithm that benefits from the fully local conjugacy of the model. We conduct extensive experiments on several real world datasets, which demonstrate that our model achieves comparable or improved performance in terms of both perplexity and topic coherence, particularly in handling short text data.
    Original languageEnglish
    Title of host publication2017 Ninth Asian Conference on Machine Learning, ACML 2017
    Subtitle of host publication15-17 November 2017, Seoul, Korea, Proceedings
    EditorsMin-Ling Zhang, Yung-Kyun Noh
    Place of PublicationUSA
    PublisherProceedings of Machine Learning Research (PMLR)
    Number of pages16
    Publication statusPublished - 2017
    EventAsian Conference on Machine Learning 2017 - Yonsei University, Seoul, Korea, Republic of (South)
    Duration: 15 Nov 201717 Nov 2017
    Conference number: 9th
    http://proceedings.mlr.press/v77/ (Proceedings)

    Publication series

    NameProceedings of Machine Learning Research
    PublisherProceedings of Machine Learning Research (PMLR)
    ISSN (Print)1938-7228


    ConferenceAsian Conference on Machine Learning 2017
    Abbreviated titleACML 2017
    Country/TerritoryKorea, Republic of (South)
    Internet address

    Cite this