Combining deep generative models and multi-lingual pretraining for semi-supervised document classification

Yi Zhu, Ehsan Shareghi, Yingzhen Li, Roi Reichart, Anna Korhonen

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP. Nonetheless, their development has happened in isolation, while the combination of both could potentially be effective for tackling task-specific labelled data shortage. To bridge this gap, we combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task. Compared to strong supervised learning baselines, our semi-supervised classification framework is highly competitive and outperforms the state-of-the-art counterparts in lowresource settings across several languages.
Original languageEnglish
Title of host publicationEACL 2021, The 16th Conference of the European Chapter of the Association for Computational Linguistics
Subtitle of host publicationProceedings of the Conference
EditorsReut Tsarfaty, Jörg Tiedemann
Place of PublicationStroudsburg PA USA
PublisherAssociation for Computational Linguistics (ACL)
Pages894-908
Number of pages15
ISBN (Electronic)9781954085022
Publication statusPublished - 2021
EventEuropean Association of Computational Linguistics Conference 2021 - Virtual, United States of America
Duration: 19 Apr 202123 Apr 2021
Conference number: 16th
https://www.aclweb.org/anthology/volumes/2021.eacl-main/ (Proceedings)
https://2021.eacl.org/ (Website)

Conference

ConferenceEuropean Association of Computational Linguistics Conference 2021
Abbreviated titleEACL 2021
CountryUnited States of America
Period19/04/2123/04/21
Internet address

Cite this