Incorporating structural alignment biases into an attentional neural translation model

Trevor Cohn; Cong Duy Vu Hoang; Ekaterina Vymolova; Kaisheng Yao; Chris Dyer; Gholamreza Haffari

Incorporating structural alignment biases into an attentional neural translation model

Trevor Cohn, Cong Duy Vu Hoang, Ekaterina Vymolova, Kaisheng Yao, Chris Dyer, Gholamreza Haffari

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

Abstract

Neural encoder-decoder models of machine translation have achieved impressive results, rivalling traditional translation models. However their modelling formulation is overly simplistic, and omits several key inductive biases built into traditional models. In this paper we extend the attentional neural translation model to include structural biases from word based alignment models, including positional bias, Markov conditioning, fertility and agreement over translation directions. We show improvements over a baseline attentional model and standard phrase-based model over several language pairs, evaluating on difficult languages in a low resource setting.

Original language	English
Title of host publication	The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016)
Subtitle of host publication	Proceedings of the Conference, June 12-17, 2016, San Diego, California, USA
Editors	Ani Nenkova, Owen Rambow
Place of Publication	Stroudsburg, PA
Publisher	Association for Computational Linguistics (ACL)
Pages	876-885
Number of pages	10
ISBN (Print)	9781941643914
Publication status	Published - 2016
Event	North American Association for Computational Linguistics 2016: Human Language Technologies - Sheraton San Diego Hotel & Marina, San Diego, United States of America Duration: 12 Jun 2016 → 17 Jun 2016 Conference number: 15th http://naacl.org/naacl-hlt-2016/

Conference

Conference	North American Association for Computational Linguistics 2016
Abbreviated title	NAACL HLT 2016
Country	United States of America
City	San Diego
Period	12/06/16 → 17/06/16
Internet address	http://naacl.org/naacl-hlt-2016/

Access to Document

Cite this

Cohn, T., Hoang, C. D. V., Vymolova, E., Yao, K., Dyer, C., & Haffari, G. (2016). Incorporating structural alignment biases into an attentional neural translation model. In A. Nenkova, & O. Rambow (Eds.), The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016): Proceedings of the Conference, June 12-17, 2016, San Diego, California, USA (pp. 876-885). Association for Computational Linguistics (ACL). http://naacl.org/naacl-hlt-2016/accepted_papers.html

Cohn, Trevor ; Hoang, Cong Duy Vu ; Vymolova, Ekaterina ; Yao, Kaisheng ; Dyer, Chris ; Haffari, Gholamreza. / Incorporating structural alignment biases into an attentional neural translation model. The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016): Proceedings of the Conference, June 12-17, 2016, San Diego, California, USA. editor / Ani Nenkova ; Owen Rambow. Stroudsburg, PA : Association for Computational Linguistics (ACL), 2016. pp. 876-885

@inproceedings{a1c1818479bf4172a6a554a208428c5e,

title = "Incorporating structural alignment biases into an attentional neural translation model",

abstract = "Neural encoder-decoder models of machine translation have achieved impressive results, rivalling traditional translation models. However their modelling formulation is overly simplistic, and omits several key inductive biases built into traditional models. In this paper we extend the attentional neural translation model to include structural biases from word based alignment models, including positional bias, Markov conditioning, fertility and agreement over translation directions. We show improvements over a baseline attentional model and standard phrase-based model over several language pairs, evaluating on difficult languages in a low resource setting.",

author = "Trevor Cohn and Hoang, {Cong Duy Vu} and Ekaterina Vymolova and Kaisheng Yao and Chris Dyer and Gholamreza Haffari",

year = "2016",

language = "English",

isbn = "9781941643914",

pages = "876--885",

editor = "Nenkova, {Ani } and Rambow, {Owen }",

booktitle = "The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016)",

publisher = "Association for Computational Linguistics (ACL)",

note = "North American Association for Computational Linguistics 2016 : Human Language Technologies, NAACL HLT 2016 ; Conference date: 12-06-2016 Through 17-06-2016",

url = "http://naacl.org/naacl-hlt-2016/",

}

Cohn, T, Hoang, CDV, Vymolova, E, Yao, K, Dyer, C & Haffari, G 2016, Incorporating structural alignment biases into an attentional neural translation model. in A Nenkova & O Rambow (eds), The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016): Proceedings of the Conference, June 12-17, 2016, San Diego, California, USA. Association for Computational Linguistics (ACL), Stroudsburg, PA, pp. 876-885, North American Association for Computational Linguistics 2016, San Diego, United States of America, 12/06/16. <http://naacl.org/naacl-hlt-2016/accepted_papers.html>

Incorporating structural alignment biases into an attentional neural translation model. / Cohn, Trevor; Hoang, Cong Duy Vu; Vymolova, Ekaterina; Yao, Kaisheng; Dyer, Chris; Haffari, Gholamreza.

The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016): Proceedings of the Conference, June 12-17, 2016, San Diego, California, USA. ed. / Ani Nenkova; Owen Rambow. Stroudsburg, PA : Association for Computational Linguistics (ACL), 2016. p. 876-885.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Incorporating structural alignment biases into an attentional neural translation model

AU - Cohn, Trevor

AU - Hoang, Cong Duy Vu

AU - Vymolova, Ekaterina

AU - Yao, Kaisheng

AU - Dyer, Chris

AU - Haffari, Gholamreza

N1 - Conference code: 15th

PY - 2016

Y1 - 2016

N2 - Neural encoder-decoder models of machine translation have achieved impressive results, rivalling traditional translation models. However their modelling formulation is overly simplistic, and omits several key inductive biases built into traditional models. In this paper we extend the attentional neural translation model to include structural biases from word based alignment models, including positional bias, Markov conditioning, fertility and agreement over translation directions. We show improvements over a baseline attentional model and standard phrase-based model over several language pairs, evaluating on difficult languages in a low resource setting.

AB - Neural encoder-decoder models of machine translation have achieved impressive results, rivalling traditional translation models. However their modelling formulation is overly simplistic, and omits several key inductive biases built into traditional models. In this paper we extend the attentional neural translation model to include structural biases from word based alignment models, including positional bias, Markov conditioning, fertility and agreement over translation directions. We show improvements over a baseline attentional model and standard phrase-based model over several language pairs, evaluating on difficult languages in a low resource setting.

UR - http://www.scopus.com/inward/record.url?scp=84994138282&partnerID=8YFLogxK

M3 - Conference Paper

AN - SCOPUS:84994138282

SN - 9781941643914

SP - 876

EP - 885

BT - The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016)

A2 - Nenkova, Ani

A2 - Rambow, Owen

PB - Association for Computational Linguistics (ACL)

CY - Stroudsburg, PA

T2 - North American Association for Computational Linguistics 2016

Y2 - 12 June 2016 through 17 June 2016

ER -

Cohn T, Hoang CDV, Vymolova E, Yao K, Dyer C, Haffari G. Incorporating structural alignment biases into an attentional neural translation model. In Nenkova A, Rambow O, editors, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016): Proceedings of the Conference, June 12-17, 2016, San Diego, California, USA. Stroudsburg, PA: Association for Computational Linguistics (ACL). 2016. p. 876-885