Multilingual neural machine translation with soft decoupled encoding

Xinyi Wang, Hieu Pham, Philip Arthur, Graham Neubig

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Multilingual training of neural machine translation (NMT) systems has led to impressive accuracy improvements on low-resource languages. However, there are still significant challenges in efficiently learning word representations in the face of paucity of data. In this paper, we propose Soft Decoupled Encoding (SDE), a multilingual lexicon encoding framework specifically designed to share lexical-level information intelligently without requiring heuristic preprocessing such as pre-segmenting the data. SDE represents a word by its spelling through a character encoding, and its semantic meaning through a latent embedding space shared by all languages. Experiments on a standard dataset of four low-resource languages show consistent improvements over strong multilingual NMT baselines, with gains of up to 2 BLEU on one of the tested languages, achieving the new state-of-the-art on all four language pairs.

Original languageEnglish
Title of host publicationInternational Conference on Learning Representations 2019
EditorsAlexander Rush
Place of PublicationLa Jolla CA USA
PublisherInternational Conference on Learning Representations (ICLR)
Number of pages13
ISBN (Print)9783800743629
Publication statusPublished - 2019
EventInternational Conference on Learning Representations 2019 - Ernest N. Morial Convention Center, New Orleans, United States of America
Duration: 6 May 20199 May 2019
Conference number: 7th
https://iclr.cc/Conferences/2019
https://openreview.net/group?id=ICLR.cc/2019/Conference (Proceedings)

Conference

ConferenceInternational Conference on Learning Representations 2019
Abbreviated titleICLR 2019
Country/TerritoryUnited States of America
CityNew Orleans
Period6/05/199/05/19
OtherThe International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning.

ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.
Internet address

Cite this