MetaLDA: A topic model that efficiently incorporates meta information

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

    Abstract

    Besides the text content, documents and their associated words usually come with rich sets of meta information, such as categories of documents and semantic/syntactic features of words, like those encoded in word embeddings. Incorporating such meta information directly into the generative process of topic models can improve modelling accuracy and topic quality, especially in the case where the word-occurrence information in the training data is insufficient. In this paper, we present a topic model, called MetaLDA, which is able to leverage either document or word meta information, or both of them jointly. With two data argumentation techniques, we can derive an efficient Gibbs sampling algorithm, which benefits from the fully local conjugacy of the model. Moreover, the algorithm is favoured by the sparsity of the meta information. Extensive experiments on several real world datasets demonstrate that our model achieves comparable or improved performance in terms of both perplexity and topic quality, particularly in handling sparse texts. In addition, compared with other models using meta information, our model runs significantly faster.

    Original languageEnglish
    Title of host publicationProceedings - 17th IEEE International Conference on Data Mining, ICDM 2017
    EditorsVijay Raghavan, Srinivas Aluru, George Karypis, Lucio Miele, Xindong Wu
    Place of PublicationPiscataway USA
    PublisherIEEE, Institute of Electrical and Electronics Engineers
    Pages635-644
    Number of pages10
    Volume2017-November
    ISBN (Print)9781538638347
    DOIs
    Publication statusPublished - 15 Dec 2017
    EventIEEE International Conference on Data Mining 2017 - New Orleans, United States of America
    Duration: 18 Nov 201721 Nov 2017
    Conference number: 17th
    http://icdm2017.bigke.org/

    Conference

    ConferenceIEEE International Conference on Data Mining 2017
    Abbreviated titleICDM 2017
    CountryUnited States of America
    CityNew Orleans
    Period18/11/1721/11/17
    Internet address

    Keywords

    • Meta information
    • Short texts
    • Topic models

    Cite this

    Zhao, H., Du, L., Buntine, W., & Liu, G. (2017). MetaLDA: A topic model that efficiently incorporates meta information. In V. Raghavan, S. Aluru, G. Karypis, L. Miele, & X. Wu (Eds.), Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017 (Vol. 2017-November, pp. 635-644). Piscataway USA: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICDM.2017.73
    Zhao, He ; Du, Lan ; Buntine, Wray ; Liu, Gang. / MetaLDA : A topic model that efficiently incorporates meta information. Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. editor / Vijay Raghavan ; Srinivas Aluru ; George Karypis ; Lucio Miele ; Xindong Wu. Vol. 2017-November Piscataway USA : IEEE, Institute of Electrical and Electronics Engineers, 2017. pp. 635-644
    @inproceedings{7be4a22191cc43039a54be373eba67c4,
    title = "MetaLDA: A topic model that efficiently incorporates meta information",
    abstract = "Besides the text content, documents and their associated words usually come with rich sets of meta information, such as categories of documents and semantic/syntactic features of words, like those encoded in word embeddings. Incorporating such meta information directly into the generative process of topic models can improve modelling accuracy and topic quality, especially in the case where the word-occurrence information in the training data is insufficient. In this paper, we present a topic model, called MetaLDA, which is able to leverage either document or word meta information, or both of them jointly. With two data argumentation techniques, we can derive an efficient Gibbs sampling algorithm, which benefits from the fully local conjugacy of the model. Moreover, the algorithm is favoured by the sparsity of the meta information. Extensive experiments on several real world datasets demonstrate that our model achieves comparable or improved performance in terms of both perplexity and topic quality, particularly in handling sparse texts. In addition, compared with other models using meta information, our model runs significantly faster.",
    keywords = "Meta information, Short texts, Topic models",
    author = "He Zhao and Lan Du and Wray Buntine and Gang Liu",
    year = "2017",
    month = "12",
    day = "15",
    doi = "10.1109/ICDM.2017.73",
    language = "English",
    isbn = "9781538638347",
    volume = "2017-November",
    pages = "635--644",
    editor = "Vijay Raghavan and Srinivas Aluru and George Karypis and Lucio Miele and Xindong Wu",
    booktitle = "Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017",
    publisher = "IEEE, Institute of Electrical and Electronics Engineers",
    address = "United States of America",

    }

    Zhao, H, Du, L, Buntine, W & Liu, G 2017, MetaLDA: A topic model that efficiently incorporates meta information. in V Raghavan, S Aluru, G Karypis, L Miele & X Wu (eds), Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. vol. 2017-November, IEEE, Institute of Electrical and Electronics Engineers, Piscataway USA, pp. 635-644, IEEE International Conference on Data Mining 2017, New Orleans, United States of America, 18/11/17. https://doi.org/10.1109/ICDM.2017.73

    MetaLDA : A topic model that efficiently incorporates meta information. / Zhao, He; Du, Lan; Buntine, Wray; Liu, Gang.

    Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. ed. / Vijay Raghavan; Srinivas Aluru; George Karypis; Lucio Miele; Xindong Wu. Vol. 2017-November Piscataway USA : IEEE, Institute of Electrical and Electronics Engineers, 2017. p. 635-644.

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

    TY - GEN

    T1 - MetaLDA

    T2 - A topic model that efficiently incorporates meta information

    AU - Zhao, He

    AU - Du, Lan

    AU - Buntine, Wray

    AU - Liu, Gang

    PY - 2017/12/15

    Y1 - 2017/12/15

    N2 - Besides the text content, documents and their associated words usually come with rich sets of meta information, such as categories of documents and semantic/syntactic features of words, like those encoded in word embeddings. Incorporating such meta information directly into the generative process of topic models can improve modelling accuracy and topic quality, especially in the case where the word-occurrence information in the training data is insufficient. In this paper, we present a topic model, called MetaLDA, which is able to leverage either document or word meta information, or both of them jointly. With two data argumentation techniques, we can derive an efficient Gibbs sampling algorithm, which benefits from the fully local conjugacy of the model. Moreover, the algorithm is favoured by the sparsity of the meta information. Extensive experiments on several real world datasets demonstrate that our model achieves comparable or improved performance in terms of both perplexity and topic quality, particularly in handling sparse texts. In addition, compared with other models using meta information, our model runs significantly faster.

    AB - Besides the text content, documents and their associated words usually come with rich sets of meta information, such as categories of documents and semantic/syntactic features of words, like those encoded in word embeddings. Incorporating such meta information directly into the generative process of topic models can improve modelling accuracy and topic quality, especially in the case where the word-occurrence information in the training data is insufficient. In this paper, we present a topic model, called MetaLDA, which is able to leverage either document or word meta information, or both of them jointly. With two data argumentation techniques, we can derive an efficient Gibbs sampling algorithm, which benefits from the fully local conjugacy of the model. Moreover, the algorithm is favoured by the sparsity of the meta information. Extensive experiments on several real world datasets demonstrate that our model achieves comparable or improved performance in terms of both perplexity and topic quality, particularly in handling sparse texts. In addition, compared with other models using meta information, our model runs significantly faster.

    KW - Meta information

    KW - Short texts

    KW - Topic models

    UR - http://www.scopus.com/inward/record.url?scp=85043981944&partnerID=8YFLogxK

    U2 - 10.1109/ICDM.2017.73

    DO - 10.1109/ICDM.2017.73

    M3 - Conference Paper

    SN - 9781538638347

    VL - 2017-November

    SP - 635

    EP - 644

    BT - Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017

    A2 - Raghavan, Vijay

    A2 - Aluru, Srinivas

    A2 - Karypis, George

    A2 - Miele, Lucio

    A2 - Wu, Xindong

    PB - IEEE, Institute of Electrical and Electronics Engineers

    CY - Piscataway USA

    ER -

    Zhao H, Du L, Buntine W, Liu G. MetaLDA: A topic model that efficiently incorporates meta information. In Raghavan V, Aluru S, Karypis G, Miele L, Wu X, editors, Proceedings - 17th IEEE International Conference on Data Mining, ICDM 2017. Vol. 2017-November. Piscataway USA: IEEE, Institute of Electrical and Electronics Engineers. 2017. p. 635-644 https://doi.org/10.1109/ICDM.2017.73