Towards decoding as continuous optimisation in neural machine translation

Cong Duy Vu Hoang, Gholamreza Haffari, Trevor Cohn

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    Abstract

    We propose a novel decoding approach for neural machine translation (NMT) based on continuous optimisation. We reformulate decoding, a discrete optimization problem, into a continuous problem, such that optimization can make use of efficient gradient-based techniques. Our powerful decoding framework allows for more accurate decoding for standard neural machine
    translation models, as well as enabling decoding in intractable models such
    as intersection of several different NMT models. Our empirical results show that
    our decoding framework is effective, and can leads to substantial improvements in translations, especially in situations where greedy search and beam search are not feasible. Finally, we show how the technique is highly competitive with, and complementary
    to, reranking
    Original languageEnglish
    Title of host publicationThe Conference on Empirical Methods in Natural Language Processing
    Subtitle of host publicationProceedings of the Conference - September 9-11, 2017, Copenhagen, Denmark
    EditorsRebecca Hwa, Sebastian Riedel
    Place of PublicationStroudsburg PA USA
    PublisherAssociation for Computational Linguistics (ACL)
    Pages146-156
    Number of pages11
    ISBN (Print)9781945626838
    DOIs
    Publication statusPublished - 2017
    EventEmpirical Methods in Natural Language Processing 2017 - Copenhagen, Denmark
    Duration: 9 Sep 201711 Sep 2017
    http://www.aclweb.org/anthology/D/D17/

    Conference

    ConferenceEmpirical Methods in Natural Language Processing 2017
    Abbreviated titleEMNLP 2017
    CountryDenmark
    CityCopenhagen
    Period9/09/1711/09/17
    Internet address

    Cite this

    Hoang, C. D. V., Haffari, G., & Cohn, T. (2017). Towards decoding as continuous optimisation in neural machine translation. In R. Hwa, & S. Riedel (Eds.), The Conference on Empirical Methods in Natural Language Processing: Proceedings of the Conference - September 9-11, 2017, Copenhagen, Denmark (pp. 146-156). Stroudsburg PA USA: Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/D17-1014
    Hoang, Cong Duy Vu ; Haffari, Gholamreza ; Cohn, Trevor. / Towards decoding as continuous optimisation in neural machine translation. The Conference on Empirical Methods in Natural Language Processing: Proceedings of the Conference - September 9-11, 2017, Copenhagen, Denmark. editor / Rebecca Hwa ; Sebastian Riedel. Stroudsburg PA USA : Association for Computational Linguistics (ACL), 2017. pp. 146-156
    @inproceedings{6d118b9b6f8841ec8c2e6d9b39248256,
    title = "Towards decoding as continuous optimisation in neural machine translation",
    abstract = "We propose a novel decoding approach for neural machine translation (NMT) based on continuous optimisation. We reformulate decoding, a discrete optimization problem, into a continuous problem, such that optimization can make use of efficient gradient-based techniques. Our powerful decoding framework allows for more accurate decoding for standard neural machinetranslation models, as well as enabling decoding in intractable models suchas intersection of several different NMT models. Our empirical results show thatour decoding framework is effective, and can leads to substantial improvements in translations, especially in situations where greedy search and beam search are not feasible. Finally, we show how the technique is highly competitive with, and complementaryto, reranking",
    author = "Hoang, {Cong Duy Vu} and Gholamreza Haffari and Trevor Cohn",
    year = "2017",
    doi = "10.18653/v1/D17-1014",
    language = "English",
    isbn = "9781945626838",
    pages = "146--156",
    editor = "Rebecca Hwa and Sebastian Riedel",
    booktitle = "The Conference on Empirical Methods in Natural Language Processing",
    publisher = "Association for Computational Linguistics (ACL)",

    }

    Hoang, CDV, Haffari, G & Cohn, T 2017, Towards decoding as continuous optimisation in neural machine translation. in R Hwa & S Riedel (eds), The Conference on Empirical Methods in Natural Language Processing: Proceedings of the Conference - September 9-11, 2017, Copenhagen, Denmark. Association for Computational Linguistics (ACL), Stroudsburg PA USA, pp. 146-156, Empirical Methods in Natural Language Processing 2017, Copenhagen, Denmark, 9/09/17. https://doi.org/10.18653/v1/D17-1014

    Towards decoding as continuous optimisation in neural machine translation. / Hoang, Cong Duy Vu; Haffari, Gholamreza; Cohn, Trevor.

    The Conference on Empirical Methods in Natural Language Processing: Proceedings of the Conference - September 9-11, 2017, Copenhagen, Denmark. ed. / Rebecca Hwa; Sebastian Riedel. Stroudsburg PA USA : Association for Computational Linguistics (ACL), 2017. p. 146-156.

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    TY - GEN

    T1 - Towards decoding as continuous optimisation in neural machine translation

    AU - Hoang, Cong Duy Vu

    AU - Haffari, Gholamreza

    AU - Cohn, Trevor

    PY - 2017

    Y1 - 2017

    N2 - We propose a novel decoding approach for neural machine translation (NMT) based on continuous optimisation. We reformulate decoding, a discrete optimization problem, into a continuous problem, such that optimization can make use of efficient gradient-based techniques. Our powerful decoding framework allows for more accurate decoding for standard neural machinetranslation models, as well as enabling decoding in intractable models suchas intersection of several different NMT models. Our empirical results show thatour decoding framework is effective, and can leads to substantial improvements in translations, especially in situations where greedy search and beam search are not feasible. Finally, we show how the technique is highly competitive with, and complementaryto, reranking

    AB - We propose a novel decoding approach for neural machine translation (NMT) based on continuous optimisation. We reformulate decoding, a discrete optimization problem, into a continuous problem, such that optimization can make use of efficient gradient-based techniques. Our powerful decoding framework allows for more accurate decoding for standard neural machinetranslation models, as well as enabling decoding in intractable models suchas intersection of several different NMT models. Our empirical results show thatour decoding framework is effective, and can leads to substantial improvements in translations, especially in situations where greedy search and beam search are not feasible. Finally, we show how the technique is highly competitive with, and complementaryto, reranking

    U2 - 10.18653/v1/D17-1014

    DO - 10.18653/v1/D17-1014

    M3 - Conference Paper

    SN - 9781945626838

    SP - 146

    EP - 156

    BT - The Conference on Empirical Methods in Natural Language Processing

    A2 - Hwa, Rebecca

    A2 - Riedel, Sebastian

    PB - Association for Computational Linguistics (ACL)

    CY - Stroudsburg PA USA

    ER -

    Hoang CDV, Haffari G, Cohn T. Towards decoding as continuous optimisation in neural machine translation. In Hwa R, Riedel S, editors, The Conference on Empirical Methods in Natural Language Processing: Proceedings of the Conference - September 9-11, 2017, Copenhagen, Denmark. Stroudsburg PA USA: Association for Computational Linguistics (ACL). 2017. p. 146-156 https://doi.org/10.18653/v1/D17-1014