Domain generalisation of NMT: fusing adapters with leave-one-domain-out training

Thuy-Trang Vu; Shahram Khadivi; Dinh Phung; Gholamreza Haffari

doi:10.18653/v1/2022.findings-acl.49

Domain generalisation of NMT: fusing adapters with leave-one-domain-out training

Thuy-Trang Vu, Shahram Khadivi, Dinh Phung, Gholamreza Haffari

Department of Data Science & AI

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

8 Citations (Scopus)

Abstract

Generalising to unseen domains is under-explored and remains a challenge in neural machine translation. Inspired by recent research in parameter-efficient transfer learning from pretrained models, this paper proposes a fusion-based generalisation method that learns to combine domain-specific parameters. We propose a leave-one-domain-out training strategy to avoid information leaking to address the challenge of not knowing the test domain during training time. Empirical results on three language pairs show that our proposed fusion method outperforms other baselines up to +0.8 BLEU score on average.

Original language	English
Title of host publication	ACL 2022 - The 60th Annual Meeting of the Association for Computational Linguistics - Findings of ACL 2022
Editors	Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Place of Publication	Stroudsburg PA USA
Publisher	Association for Computational Linguistics (ACL)
Pages	582-588
Number of pages	7
ISBN (Electronic)	9781955917254
DOIs	https://doi.org/10.18653/v1/2022.findings-acl.49
Publication status	Published - 2022
Event	Annual Meeting of the Association of Computational Linguistics 2022 - Dublin, Ireland Duration: 22 May 2022 → 27 May 2022 Conference number: 60th https://aclanthology.org/volumes/2022.acl-short/ (Proceedings - Short) https://aclanthology.org/volumes/2022.acl-long/ (Proceedings - Long) https://www.2022.aclweb.org/ (Website)

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
Publisher	Association for Computational Linguistics (ACL)
ISSN (Print)	0736-587X

Conference

Conference	Annual Meeting of the Association of Computational Linguistics 2022
Abbreviated title	ACL 2022
Country/Territory	Ireland
City	Dublin
Period	22/05/22 → 27/05/22
Internet address	https://aclanthology.org/volumes/2022.acl-short/ (Proceedings - Short) https://aclanthology.org/volumes/2022.acl-long/ (Proceedings - Long) https://www.2022.aclweb.org/ (Website)

Access to Document

10.18653/v1/2022.findings-acl.49

423777732-oaFinal published version, 334 KB

Cite this

Vu, T.-T., Khadivi, S., Phung, D., & Haffari, G. (2022). Domain generalisation of NMT: fusing adapters with leave-one-domain-out training. In S. Muresan, P. Nakov, & A. Villavicencio (Eds.), ACL 2022 - The 60th Annual Meeting of the Association for Computational Linguistics - Findings of ACL 2022 (pp. 582-588). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2022.findings-acl.49

Vu, Thuy-Trang ; Khadivi, Shahram ; Phung, Dinh et al. / Domain generalisation of NMT : fusing adapters with leave-one-domain-out training. ACL 2022 - The 60th Annual Meeting of the Association for Computational Linguistics - Findings of ACL 2022. editor / Smaranda Muresan ; Preslav Nakov ; Aline Villavicencio. Stroudsburg PA USA : Association for Computational Linguistics (ACL), 2022. pp. 582-588 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{262f763aa721446a96172c7cbce660aa,

title = "Domain generalisation of NMT: fusing adapters with leave-one-domain-out training",

abstract = "Generalising to unseen domains is under-explored and remains a challenge in neural machine translation. Inspired by recent research in parameter-efficient transfer learning from pretrained models, this paper proposes a fusion-based generalisation method that learns to combine domain-specific parameters. We propose a leave-one-domain-out training strategy to avoid information leaking to address the challenge of not knowing the test domain during training time. Empirical results on three language pairs show that our proposed fusion method outperforms other baselines up to +0.8 BLEU score on average.",

author = "Thuy-Trang Vu and Shahram Khadivi and Dinh Phung and Gholamreza Haffari",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; Annual Meeting of the Association of Computational Linguistics 2022, ACL 2022 ; Conference date: 22-05-2022 Through 27-05-2022",

year = "2022",

doi = "10.18653/v1/2022.findings-acl.49",

language = "English",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "582--588",

editor = "Smaranda Muresan and Preslav Nakov and Aline Villavicencio",

booktitle = "ACL 2022 - The 60th Annual Meeting of the Association for Computational Linguistics - Findings of ACL 2022",

url = "https://aclanthology.org/volumes/2022.acl-short/, https://aclanthology.org/volumes/2022.acl-long/, https://www.2022.aclweb.org/",

}

Vu, T-T, Khadivi, S, Phung, D & Haffari, G 2022, Domain generalisation of NMT: fusing adapters with leave-one-domain-out training. in S Muresan, P Nakov & A Villavicencio (eds), ACL 2022 - The 60th Annual Meeting of the Association for Computational Linguistics - Findings of ACL 2022. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), Stroudsburg PA USA, pp. 582-588, Annual Meeting of the Association of Computational Linguistics 2022, Dublin, Ireland, 22/05/22. https://doi.org/10.18653/v1/2022.findings-acl.49

Domain generalisation of NMT: fusing adapters with leave-one-domain-out training. / Vu, Thuy-Trang; Khadivi, Shahram; Phung, Dinh et al.
ACL 2022 - The 60th Annual Meeting of the Association for Computational Linguistics - Findings of ACL 2022. ed. / Smaranda Muresan; Preslav Nakov; Aline Villavicencio. Stroudsburg PA USA: Association for Computational Linguistics (ACL), 2022. p. 582-588 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Domain generalisation of NMT

T2 - Annual Meeting of the Association of Computational Linguistics 2022

AU - Vu, Thuy-Trang

AU - Khadivi, Shahram

AU - Phung, Dinh

AU - Haffari, Gholamreza

N1 - Conference code: 60th

PY - 2022

Y1 - 2022

N2 - Generalising to unseen domains is under-explored and remains a challenge in neural machine translation. Inspired by recent research in parameter-efficient transfer learning from pretrained models, this paper proposes a fusion-based generalisation method that learns to combine domain-specific parameters. We propose a leave-one-domain-out training strategy to avoid information leaking to address the challenge of not knowing the test domain during training time. Empirical results on three language pairs show that our proposed fusion method outperforms other baselines up to +0.8 BLEU score on average.

AB - Generalising to unseen domains is under-explored and remains a challenge in neural machine translation. Inspired by recent research in parameter-efficient transfer learning from pretrained models, this paper proposes a fusion-based generalisation method that learns to combine domain-specific parameters. We propose a leave-one-domain-out training strategy to avoid information leaking to address the challenge of not knowing the test domain during training time. Empirical results on three language pairs show that our proposed fusion method outperforms other baselines up to +0.8 BLEU score on average.

UR - http://www.scopus.com/inward/record.url?scp=85139889628&partnerID=8YFLogxK

U2 - 10.18653/v1/2022.findings-acl.49

DO - 10.18653/v1/2022.findings-acl.49

M3 - Conference Paper

AN - SCOPUS:85139889628

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 582

EP - 588

BT - ACL 2022 - The 60th Annual Meeting of the Association for Computational Linguistics - Findings of ACL 2022

A2 - Muresan, Smaranda

A2 - Nakov, Preslav

A2 - Villavicencio, Aline

PB - Association for Computational Linguistics (ACL)

CY - Stroudsburg PA USA

Y2 - 22 May 2022 through 27 May 2022

ER -

Vu TT, Khadivi S, Phung D , Haffari G. Domain generalisation of NMT: fusing adapters with leave-one-domain-out training. In Muresan S, Nakov P, Villavicencio A, editors, ACL 2022 - The 60th Annual Meeting of the Association for Computational Linguistics - Findings of ACL 2022. Stroudsburg PA USA: Association for Computational Linguistics (ACL). 2022. p. 582-588. (Proceedings of the Annual Meeting of the Association for Computational Linguistics). doi: 10.18653/v1/2022.findings-acl.49

Domain generalisation of NMT: fusing adapters with leave-one-domain-out training

Abstract

Publication series

Conference

Access to Document

Other files and links

Exploiting Context in Multilingual Understanding and Generation

Cite this

Domain generalisation of NMT: fusing adapters with leave-one-domain-out training

Abstract

Publication series

Conference

Access to Document

Other files and links

Projects

Exploiting Context in Multilingual Understanding and Generation

Cite this