Neural machine translation via binary code prediction

Yusuke Oda, Philip Arthur, Graham Neubig, Koichiro Yoshino, Satoshi Nakamura

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

5 Citations (Scopus)

Abstract

In this paper, we propose a new method for calculating the output layer in neural machine translation systems. The method is based on predicting a binary code for each word and can reduce computation time/memory requirements of the output layer to be logarithmic in vocabulary size in the best case. In addition, we also introduce two advanced approaches to improve the robustness of the proposed model: using error-correcting codes and combining softmax and binary codes. Experiments on two English ? Japanese bidirectional translation tasks show proposed models achieve BLEU scores that approach the softmax, while reducing memory usage to the order of less than 1/10 and improving decoding speed on CPUs by x5 to x10.

Original languageEnglish
Title of host publicationACL 2017 - The 55th Annual Meeting of the Association for Computational Linguistics
Subtitle of host publicationProceedings of the Conference, Vol. 1 (Long Papers) July 30
EditorsRegina Barzilay, Min-Yen Kan
Place of PublicationStroudsburg PA USA
PublisherAssociation for Computational Linguistics (ACL)
Pages850-860
Number of pages11
Volume1
ISBN (Electronic)9781945626753
DOIs
Publication statusPublished - 2017
Externally publishedYes
EventAnnual Meeting of the Association of Computational Linguistics 2017 - Vancouver, Canada
Duration: 30 Jul 20174 Aug 2017
Conference number: 55th
https://www.aclweb.org/anthology/events/acl-2017/ (Proceedings)

Conference

ConferenceAnnual Meeting of the Association of Computational Linguistics 2017
Abbreviated titleACL 2017
CountryCanada
CityVancouver
Period30/07/174/08/17
Internet address

Cite this