A closer look at few-shot crosslingual transfer: the choice of shots matters

Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, Hinrich Schutze

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Few-shot crosslingual transfer has been shown to outperform its zero-shot counterpart with pretrained encoders like multilingual BERT. Despite its growing popularity, little to no attention has been paid to standardizing and analyzing the design of few-shot experiments. In this work, we highlight a fundamental risk posed by this shortcoming, illustrating that the model exhibits a high degree of sensitivity to the selection of few shots. We conduct a large-scale experimental study on 40 sets of sampled few shots for six diverse NLP tasks across up to 40 languages. We provide an analysis of success and failure cases of few-shot transfer, which highlights the role of lexical features. Additionally, we show that a straightforward full model finetuning approach is quite effective for few-shot transfer, outperforming several state-of-the-art few-shot approaches. As a step towards standardizing few-shot crosslingual experimental designs, we make our sampled few shots publicly available.
Original languageEnglish
Title of host publicationACL-IJCNLP 2021, The 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing
Subtitle of host publicationProceedings of the Conference, Vol. 1 (Long Papers)
EditorsFei Xia, Wenjie Li, Roberto Navigli
Place of PublicationStroudsburg PA USA
PublisherAssociation for Computational Linguistics (ACL)
Pages5751–5767
Number of pages17
Volume1
ISBN (Electronic)9781954085527
DOIs
Publication statusPublished - 2021
EventAnnual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing 2021 - Online, Bangkok, Thailand
Duration: 1 Aug 20216 Aug 2021
Conference number: 59th & 11th
https://aclanthology.org/2021.acl-long.0/ (Proceedings)
https://2021.aclweb.org (Website)

Conference

ConferenceAnnual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing 2021
Abbreviated titleACL-IJCNLP 2021
CountryThailand
CityBangkok
Period1/08/216/08/21
Internet address

Cite this