Progressive Class Semantic Matching for Semi-supervised Text Classification

Hai Ming Xu, Lingqiao Liu, Ehsan Abbasnejad

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

9 Citations (Scopus)

Abstract

Semi-supervised learning is a promising way to reduce the annotation cost for text-classification. Combining with pre-trained language models (PLMs), e.g., BERT, recent semi-supervised learning methods achieved impressive performance. In this work, we further investigate the marriage between semi-supervised learning and a pre-trained language model. Unlike existing approaches that utilize PLMs only for model parameter initialization, we explore the inherent topic matching capability inside PLMs for building a more powerful semi-supervised learning approach. Specifically, we propose a joint semi-supervised learning process that can progressively build a standard K-way classifier and a matching network for the input text and the Class Semantic Representation (CSR). The CSR will be initialized from the given labeled sentences and progressively updated through the training process. By means of extensive experiments, we show that our method can not only bring remarkable improvement to baselines, but also overall be more stable, and achieves state-ofthe-art performance in semi-supervised text classification. Code is available at: https://github.com/HeimingX/PCM.

Original languageEnglish
Title of host publicationNAACL 2022, 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference
EditorsRyan Cotterell, Danilo Croce, Jordan Zhang
Place of PublicationStroudsburg PA USA
PublisherAssociation for Computational Linguistics (ACL)
Pages3003-3013
Number of pages11
ISBN (Electronic)9781955917711
DOIs
Publication statusPublished - 2022
Externally publishedYes
EventNorth American Association for Computational Linguistics 2022: NAACL 2022 - Seattle, United States of America
Duration: 10 Jul 202215 Jul 2022
https://aclanthology.org/volumes/2022.findings-naacl/

Conference

ConferenceNorth American Association for Computational Linguistics 2022
Abbreviated titleNAACL 2022
Country/TerritoryUnited States of America
CitySeattle
Period10/07/2215/07/22
Internet address

Cite this