Bayesian learning for neural dependency parsing

Ehsan Shareghi, Yingzhen Li, Yi Zhu, Roi Reichart, Anna Korhonen

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

2 Citations (Scopus)

Abstract

While neural dependency parsers provide state-of-the-art accuracy for several languages, they still rely on large amounts of costly labeled training data. We demonstrate that in the small data regime, where uncertainty around parameter estimation and model prediction matters the most, Bayesian neural modeling is very effective. In order to overcome the computational and statistical costs of the approximate inference step in this framework, we utilize an efficient sampling procedure via stochastic gradient Langevin dynamics to generate samples from the approximated posterior. Moreover, we show that our Bayesian neural parser can be further improved when integrated into a multi-task parsing and POS tagging framework, designed to minimize task interference via an adversarial procedure. When trained and tested on 6 languages with less than 5k training instances, our parser consistently outperforms the strong BiLSTM baseline (Kiperwasser and Goldberg, 2016). Compared with the BiAFFINE parser (Dozat et al., 2017) our model achieves an improvement of up to 3% for Vietnamese and Irish, while our multi-task model achieves an improvement of up to 9% across five languages: Farsi, Russian, Turkish, Vietnamese, and Irish.

Original languageEnglish
Title of host publicationNAACL 2019, The 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Subtitle of host publicationProceedings of the Conference Vol. 1 (Long and Short Papers), June 2 - June 7, 2019
EditorsChristy Doran, Thamar Solorio
Place of PublicationStroudsburg PA USA
PublisherAssociation for Computational Linguistics (ACL)
Pages3509-3519
Number of pages11
Volume1
ISBN (Electronic)9781950737130
DOIs
Publication statusPublished - Jun 2019
Externally publishedYes
EventNorth American Association for Computational Linguistics 2019: Human Language Technologies - Minneapolis, United States of America
Duration: 2 Jun 20197 Jun 2019
https://naacl2019.org/

Conference

ConferenceNorth American Association for Computational Linguistics 2019
Abbreviated titleNAACL HLT 2019
CountryUnited States of America
CityMinneapolis
Period2/06/197/06/19
Internet address

Cite this