Bayesian Multi-label Learning with Sparse Features and Labels, and Label Co-occurrences

He Zhao, Piyush Rai, Lan Du, Wray Buntine

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    Abstract

    We present a probabilistic, fully Bayesian framework for multi-label learning. Our framework is based on the idea of learning a joint low-rank embedding of the label matrix and the label co-occurrence matrix. The proposed framework has the following appealing aspects: (1) It leverages the sparsity in the label matrix and the feature matrix, which results in very efficient inference, especially for sparse datasets, commonly encountered in multi-label learning problems, and (2) By effectively utilizing the label co-occurrence information, the model yields improved prediction accuracies, especially in the case where the amount of training data is low and/or the label matrix has a significant fraction of missing labels. Our framework enjoys full local conjugacy and admits a simple inference procedure via a scalable Gibbs sampler. We report experimental results on a number of benchmark datasets, on which it outperforms several state-of-the-art multi-label learning models.
    LanguageEnglish
    Title of host publication2018 Twenty-First International Conference on Artificial Intelligence and Statistics, AISTATS 2018
    Subtitle of host publication9-11 April 2018, Lanzarote, Canary Islands, Proceedings
    EditorsAmos Storkey, Fernando Perez-Cruz
    Place of PublicationLanzarote, Canary Islands
    PublisherPMLR
    Pages1943-1951
    Number of pages9
    Volume84
    Publication statusPublished - 2018
    EventInternational Conference on Artificial Intelligence and Statistics 2018 - Playa Blanca, Lanzarote, Spain
    Duration: 9 Apr 201811 Apr 2018
    Conference number: 21st

    Publication series

    NameProceedings of Machine Learning Research
    PublisherPMLR
    Volume84
    ISSN (Print)1938-7228

    Conference

    ConferenceInternational Conference on Artificial Intelligence and Statistics 2018
    Abbreviated titleAISTATS 2018
    CountrySpain
    CityPlaya Blanca, Lanzarote
    Period9/04/1811/04/18

    Cite this

    Zhao, H., Rai, P., Du, L., & Buntine, W. (2018). Bayesian Multi-label Learning with Sparse Features and Labels, and Label Co-occurrences. In A. Storkey, & F. Perez-Cruz (Eds.), 2018 Twenty-First International Conference on Artificial Intelligence and Statistics, AISTATS 2018 : 9-11 April 2018, Lanzarote, Canary Islands, Proceedings (Vol. 84, pp. 1943-1951). (Proceedings of Machine Learning Research; Vol. 84). Lanzarote, Canary Islands: PMLR.
    Zhao, He ; Rai, Piyush ; Du, Lan ; Buntine, Wray. / Bayesian Multi-label Learning with Sparse Features and Labels, and Label Co-occurrences. 2018 Twenty-First International Conference on Artificial Intelligence and Statistics, AISTATS 2018 : 9-11 April 2018, Lanzarote, Canary Islands, Proceedings . editor / Amos Storkey ; Fernando Perez-Cruz. Vol. 84 Lanzarote, Canary Islands : PMLR, 2018. pp. 1943-1951 (Proceedings of Machine Learning Research).
    @inproceedings{c35179487d8f4a73b2a44985b535b100,
    title = "Bayesian Multi-label Learning with Sparse Features and Labels, and Label Co-occurrences",
    abstract = "We present a probabilistic, fully Bayesian framework for multi-label learning. Our framework is based on the idea of learning a joint low-rank embedding of the label matrix and the label co-occurrence matrix. The proposed framework has the following appealing aspects: (1) It leverages the sparsity in the label matrix and the feature matrix, which results in very efficient inference, especially for sparse datasets, commonly encountered in multi-label learning problems, and (2) By effectively utilizing the label co-occurrence information, the model yields improved prediction accuracies, especially in the case where the amount of training data is low and/or the label matrix has a significant fraction of missing labels. Our framework enjoys full local conjugacy and admits a simple inference procedure via a scalable Gibbs sampler. We report experimental results on a number of benchmark datasets, on which it outperforms several state-of-the-art multi-label learning models.",
    author = "He Zhao and Piyush Rai and Lan Du and Wray Buntine",
    year = "2018",
    language = "English",
    volume = "84",
    series = "Proceedings of Machine Learning Research",
    publisher = "PMLR",
    pages = "1943--1951",
    editor = "Amos Storkey and Fernando Perez-Cruz",
    booktitle = "2018 Twenty-First International Conference on Artificial Intelligence and Statistics, AISTATS 2018",

    }

    Zhao, H, Rai, P, Du, L & Buntine, W 2018, Bayesian Multi-label Learning with Sparse Features and Labels, and Label Co-occurrences. in A Storkey & F Perez-Cruz (eds), 2018 Twenty-First International Conference on Artificial Intelligence and Statistics, AISTATS 2018 : 9-11 April 2018, Lanzarote, Canary Islands, Proceedings . vol. 84, Proceedings of Machine Learning Research, vol. 84, PMLR, Lanzarote, Canary Islands, pp. 1943-1951, International Conference on Artificial Intelligence and Statistics 2018, Playa Blanca, Lanzarote, Spain, 9/04/18.

    Bayesian Multi-label Learning with Sparse Features and Labels, and Label Co-occurrences. / Zhao, He; Rai, Piyush; Du, Lan; Buntine, Wray.

    2018 Twenty-First International Conference on Artificial Intelligence and Statistics, AISTATS 2018 : 9-11 April 2018, Lanzarote, Canary Islands, Proceedings . ed. / Amos Storkey; Fernando Perez-Cruz. Vol. 84 Lanzarote, Canary Islands : PMLR, 2018. p. 1943-1951 (Proceedings of Machine Learning Research; Vol. 84).

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    TY - GEN

    T1 - Bayesian Multi-label Learning with Sparse Features and Labels, and Label Co-occurrences

    AU - Zhao, He

    AU - Rai, Piyush

    AU - Du, Lan

    AU - Buntine, Wray

    PY - 2018

    Y1 - 2018

    N2 - We present a probabilistic, fully Bayesian framework for multi-label learning. Our framework is based on the idea of learning a joint low-rank embedding of the label matrix and the label co-occurrence matrix. The proposed framework has the following appealing aspects: (1) It leverages the sparsity in the label matrix and the feature matrix, which results in very efficient inference, especially for sparse datasets, commonly encountered in multi-label learning problems, and (2) By effectively utilizing the label co-occurrence information, the model yields improved prediction accuracies, especially in the case where the amount of training data is low and/or the label matrix has a significant fraction of missing labels. Our framework enjoys full local conjugacy and admits a simple inference procedure via a scalable Gibbs sampler. We report experimental results on a number of benchmark datasets, on which it outperforms several state-of-the-art multi-label learning models.

    AB - We present a probabilistic, fully Bayesian framework for multi-label learning. Our framework is based on the idea of learning a joint low-rank embedding of the label matrix and the label co-occurrence matrix. The proposed framework has the following appealing aspects: (1) It leverages the sparsity in the label matrix and the feature matrix, which results in very efficient inference, especially for sparse datasets, commonly encountered in multi-label learning problems, and (2) By effectively utilizing the label co-occurrence information, the model yields improved prediction accuracies, especially in the case where the amount of training data is low and/or the label matrix has a significant fraction of missing labels. Our framework enjoys full local conjugacy and admits a simple inference procedure via a scalable Gibbs sampler. We report experimental results on a number of benchmark datasets, on which it outperforms several state-of-the-art multi-label learning models.

    M3 - Conference Paper

    VL - 84

    T3 - Proceedings of Machine Learning Research

    SP - 1943

    EP - 1951

    BT - 2018 Twenty-First International Conference on Artificial Intelligence and Statistics, AISTATS 2018

    A2 - Storkey, Amos

    A2 - Perez-Cruz, Fernando

    PB - PMLR

    CY - Lanzarote, Canary Islands

    ER -

    Zhao H, Rai P, Du L, Buntine W. Bayesian Multi-label Learning with Sparse Features and Labels, and Label Co-occurrences. In Storkey A, Perez-Cruz F, editors, 2018 Twenty-First International Conference on Artificial Intelligence and Statistics, AISTATS 2018 : 9-11 April 2018, Lanzarote, Canary Islands, Proceedings . Vol. 84. Lanzarote, Canary Islands: PMLR. 2018. p. 1943-1951. (Proceedings of Machine Learning Research).