Improving numerical reasoning skills in the modular approach for complex question answering on text

Xiaoyu Guo; Yuan-Fang Li; Reza Haffari

Improving numerical reasoning skills in the modular approach for complex question answering on text

Xiaoyu Guo, Yuan-Fang Li, Reza Haffari

Department of Data Science & AI

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

3 Citations (Scopus)

Abstract

Numerical reasoning skills are essential for complex question answering (CQA) over text. It requires opertaions including counting, comparison, addition and subtraction. A successful approach to CQA on text, Neural Module Networks (NMNs), follows the programmer-interpreter paradigm and leverages specialised modules to perform compositional reasoning. However, the NMNs framework does not consider the relationship between numbers and entities in both questions and paragraphs. We propose effective techniques to improve NMNs' numerical reasoning capabilities by making the interpreter questionaware and capturing the relationship between entities and numbers. On the same subset of the DROP dataset for CQA on text, experimental results show that our additions outperform the original NMNs by 3.0 points for the overall F1 score.

Original language	English
Title of host publication	Findings of the Association for Computational Linguistics: EMNLP 2021
Editors	Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Place of Publication	Stroudsburg PA USA
Publisher	Association for Computational Linguistics (ACL)
Pages	2713–2718
Number of pages	6
ISBN (Electronic)	9781955917100
Publication status	Published - 2021
Event	Empirical Methods in Natural Language Processing 2021 - Online, Punta Cana, Dominican Republic Duration: 7 Nov 2021 → 11 Nov 2021 https://2021.emnlp.org/ (Website) https://aclanthology.org/2021.emnlp-main.0/ (Proceedings) https://aclanthology.org/2021.findings-emnlp.0/ (Proceedings - findings)

Conference

Conference	Empirical Methods in Natural Language Processing 2021
Abbreviated title	EMNLP 2021
Country/Territory	Dominican Republic
City	Punta Cana
Period	7/11/21 → 11/11/21
Internet address	https://2021.emnlp.org/ (Website) https://aclanthology.org/2021.emnlp-main.0/ (Proceedings) https://aclanthology.org/2021.findings-emnlp.0/ (Proceedings - findings)

Access to Document

362146291-oaFinal published version, 502 KB

https://aclanthology.org/2021.findings-emnlp.231

Cite this

Guo, X., Li, Y.-F., & Haffari, R. (2021). Improving numerical reasoning skills in the modular approach for complex question answering on text. In X. Huang, L. Specia, & S. Wen-tau Yih (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2021 (pp. 2713–2718). Article 231 Association for Computational Linguistics (ACL). https://aclanthology.org/2021.findings-emnlp.231

Guo, Xiaoyu ; Li, Yuan-Fang ; Haffari, Reza. / Improving numerical reasoning skills in the modular approach for complex question answering on text. Findings of the Association for Computational Linguistics: EMNLP 2021. editor / Xuanjing Huang ; Lucia Specia ; Scott Wen-tau Yih. Stroudsburg PA USA : Association for Computational Linguistics (ACL), 2021. pp. 2713–2718

@inproceedings{1daaf1801d774af0891ea031a05e28c1,

title = "Improving numerical reasoning skills in the modular approach for complex question answering on text",

abstract = "Numerical reasoning skills are essential for complex question answering (CQA) over text. It requires opertaions including counting, comparison, addition and subtraction. A successful approach to CQA on text, Neural Module Networks (NMNs), follows the programmer-interpreter paradigm and leverages specialised modules to perform compositional reasoning. However, the NMNs framework does not consider the relationship between numbers and entities in both questions and paragraphs. We propose effective techniques to improve NMNs' numerical reasoning capabilities by making the interpreter questionaware and capturing the relationship between entities and numbers. On the same subset of the DROP dataset for CQA on text, experimental results show that our additions outperform the original NMNs by 3.0 points for the overall F1 score. ",

author = "Xiaoyu Guo and Yuan-Fang Li and Reza Haffari",

note = "Funding Information: This research was supported in part by the Future Fellowship FT190100039 from the Australian Research Council. The computational resources for this work were provided by the Multi-modal Australian ScienceS Imaging and Visualisation Environment (MASSIVE) (www.massive.org.au). We would like to thank the anonymous reviewers for their useful comments to improve the manuscript. Publisher Copyright: {\textcopyright} 2021 Association for Computational Linguistics.; Empirical Methods in Natural Language Processing 2021, EMNLP 2021 ; Conference date: 07-11-2021 Through 11-11-2021",

year = "2021",

language = "English",

pages = "2713–2718",

editor = "Xuanjing Huang and Lucia Specia and \{Wen-tau Yih\}, Scott",

booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",

publisher = "Association for Computational Linguistics (ACL)",

url = "https://2021.emnlp.org/, https://aclanthology.org/2021.emnlp-main.0/, https://aclanthology.org/2021.findings-emnlp.0/",

}

Guo, X, Li, Y-F & Haffari, R 2021, Improving numerical reasoning skills in the modular approach for complex question answering on text. in X Huang, L Specia & S Wen-tau Yih (eds), Findings of the Association for Computational Linguistics: EMNLP 2021., 231, Association for Computational Linguistics (ACL), Stroudsburg PA USA, pp. 2713–2718, Empirical Methods in Natural Language Processing 2021, Punta Cana, Dominican Republic, 7/11/21. <https://aclanthology.org/2021.findings-emnlp.231>

Improving numerical reasoning skills in the modular approach for complex question answering on text. / Guo, Xiaoyu; Li, Yuan-Fang ; Haffari, Reza.
Findings of the Association for Computational Linguistics: EMNLP 2021. ed. / Xuanjing Huang; Lucia Specia; Scott Wen-tau Yih. Stroudsburg PA USA: Association for Computational Linguistics (ACL), 2021. p. 2713–2718 231.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Improving numerical reasoning skills in the modular approach for complex question answering on text

AU - Guo, Xiaoyu

AU - Li, Yuan-Fang

AU - Haffari, Reza

N1 - Funding Information: This research was supported in part by the Future Fellowship FT190100039 from the Australian Research Council. The computational resources for this work were provided by the Multi-modal Australian ScienceS Imaging and Visualisation Environment (MASSIVE) (www.massive.org.au). We would like to thank the anonymous reviewers for their useful comments to improve the manuscript. Publisher Copyright: © 2021 Association for Computational Linguistics.

PY - 2021

Y1 - 2021

N2 - Numerical reasoning skills are essential for complex question answering (CQA) over text. It requires opertaions including counting, comparison, addition and subtraction. A successful approach to CQA on text, Neural Module Networks (NMNs), follows the programmer-interpreter paradigm and leverages specialised modules to perform compositional reasoning. However, the NMNs framework does not consider the relationship between numbers and entities in both questions and paragraphs. We propose effective techniques to improve NMNs' numerical reasoning capabilities by making the interpreter questionaware and capturing the relationship between entities and numbers. On the same subset of the DROP dataset for CQA on text, experimental results show that our additions outperform the original NMNs by 3.0 points for the overall F1 score.

AB - Numerical reasoning skills are essential for complex question answering (CQA) over text. It requires opertaions including counting, comparison, addition and subtraction. A successful approach to CQA on text, Neural Module Networks (NMNs), follows the programmer-interpreter paradigm and leverages specialised modules to perform compositional reasoning. However, the NMNs framework does not consider the relationship between numbers and entities in both questions and paragraphs. We propose effective techniques to improve NMNs' numerical reasoning capabilities by making the interpreter questionaware and capturing the relationship between entities and numbers. On the same subset of the DROP dataset for CQA on text, experimental results show that our additions outperform the original NMNs by 3.0 points for the overall F1 score.

UR - https://www.scopus.com/pages/publications/85129163649

M3 - Conference Paper

AN - SCOPUS:85129163649

SP - 2713

EP - 2718

BT - Findings of the Association for Computational Linguistics: EMNLP 2021

A2 - Huang, Xuanjing

A2 - Specia, Lucia

A2 - Wen-tau Yih, Scott

PB - Association for Computational Linguistics (ACL)

CY - Stroudsburg PA USA

T2 - Empirical Methods in Natural Language Processing 2021

Y2 - 7 November 2021 through 11 November 2021

ER -

Improving numerical reasoning skills in the modular approach for complex question answering on text

Abstract

Conference

Access to Document

Other files and links

Exploiting Context in Multilingual Understanding and Generation

Cite this

Improving numerical reasoning skills in the modular approach for complex question answering on text

Abstract

Conference

Access to Document

Other files and links

Projects

Exploiting Context in Multilingual Understanding and Generation

Cite this