Towards automatic content analysis of rhetorical structure in Brazilian college entrance essays

Rafael Ferreira Mello; Giuseppe Fiorentino; Péricles Miranda; Hilário Oliveira; Mladen Raković; Dragan Gašević

doi:10.1007/978-3-030-78270-2_29

Towards automatic content analysis of rhetorical structure in Brazilian college entrance essays

Rafael Ferreira Mello, Giuseppe Fiorentino, Péricles Miranda, Hilário Oliveira, Mladen Raković, Dragan Gašević

Department of Human Centred Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

9 Citations (Scopus)

Abstract

Essay scorers manually look for the presence of required rhetorical categories to evaluate coherence, which is a time-consuming task. Several attempts in the literature have been reported to automate the identification of rhetorical categories in essays with machine learning. However, existing machine learning algorithms are mostly trained on content features which can lead to over-fitting and hindering model generalizability. Thus, this paper proposed a set of content-independent features to identify rhetorical categories. The best performing classifier, XGBoost, achieved performance comparable to human annotation and outperformed previous models.

Original language	English
Title of host publication	22nd International Conference, AIED 2021 Utrecht, The Netherlands, June 14–18, 2021 Proceedings, Part II
Editors	Ido Roll, Danielle McNamara, Sergey Sosnovsky, Rose Luckin, Vania Dimitrova
Place of Publication	Cham Switzerland
Publisher	Springer
Pages	162-167
Number of pages	6
ISBN (Electronic)	9783030782702
ISBN (Print)	9783030782696
DOIs	https://doi.org/10.1007/978-3-030-78270-2_29
Publication status	Published - 2021
Event	International Conference on Artificial Intelligence in Education 2021 - Utrecht, Netherlands Duration: 14 Jun 2021 → 18 Jun 2021 Conference number: 22nd https://link.springer.com/book/10.1007/978-3-030-78292-4 (Proceedings)

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	12749
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	International Conference on Artificial Intelligence in Education 2021
Abbreviated title	AIED 2021
Country/Territory	Netherlands
City	Utrecht
Period	14/06/21 → 18/06/21
Internet address	https://link.springer.com/book/10.1007/978-3-030-78292-4 (Proceedings)

Keywords

Content analytics
Essay analysis
Rhetoric structure

Access to Document

10.1007/978-3-030-78270-2_29

Cite this

Mello, R. F., Fiorentino, G., Miranda, P., Oliveira, H., Raković, M., & Gašević, D. (2021). Towards automatic content analysis of rhetorical structure in Brazilian college entrance essays. In I. Roll, D. McNamara, S. Sosnovsky, R. Luckin, & V. Dimitrova (Eds.), 22nd International Conference, AIED 2021 Utrecht, The Netherlands, June 14–18, 2021 Proceedings, Part II (pp. 162-167). (Lecture Notes in Computer Science; Vol. 12749). Springer. https://doi.org/10.1007/978-3-030-78270-2_29

Mello, Rafael Ferreira ; Fiorentino, Giuseppe ; Miranda, Péricles et al. / Towards automatic content analysis of rhetorical structure in Brazilian college entrance essays. 22nd International Conference, AIED 2021 Utrecht, The Netherlands, June 14–18, 2021 Proceedings, Part II. editor / Ido Roll ; Danielle McNamara ; Sergey Sosnovsky ; Rose Luckin ; Vania Dimitrova. Cham Switzerland : Springer, 2021. pp. 162-167 (Lecture Notes in Computer Science).

@inproceedings{3a10a9fda81e4cb59d0cddb317671570,

title = "Towards automatic content analysis of rhetorical structure in Brazilian college entrance essays",

abstract = "Essay scorers manually look for the presence of required rhetorical categories to evaluate coherence, which is a time-consuming task. Several attempts in the literature have been reported to automate the identification of rhetorical categories in essays with machine learning. However, existing machine learning algorithms are mostly trained on content features which can lead to over-fitting and hindering model generalizability. Thus, this paper proposed a set of content-independent features to identify rhetorical categories. The best performing classifier, XGBoost, achieved performance comparable to human annotation and outperformed previous models.",

keywords = "Content analytics, Essay analysis, Rhetoric structure",

author = "Mello, \{Rafael Ferreira\} and Giuseppe Fiorentino and P{\'e}ricles Miranda and Hil{\'a}rio Oliveira and Mladen Rakovi{\'c} and Dragan Ga{\v s}evi{\'c}",

note = "Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; International Conference on Artificial Intelligence in Education 2021, AIED 2021 ; Conference date: 14-06-2021 Through 18-06-2021",

year = "2021",

doi = "10.1007/978-3-030-78270-2\_29",

language = "English",

isbn = "9783030782696",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "162--167",

editor = "Ido Roll and Danielle McNamara and Sergey Sosnovsky and Rose Luckin and Vania Dimitrova",

booktitle = "22nd International Conference, AIED 2021 Utrecht, The Netherlands, June 14–18, 2021 Proceedings, Part II",

address = "Switzerland",

url = "https://link.springer.com/book/10.1007/978-3-030-78292-4",

}

Mello, RF, Fiorentino, G, Miranda, P, Oliveira, H, Raković, M & Gašević, D 2021, Towards automatic content analysis of rhetorical structure in Brazilian college entrance essays. in I Roll, D McNamara, S Sosnovsky, R Luckin & V Dimitrova (eds), 22nd International Conference, AIED 2021 Utrecht, The Netherlands, June 14–18, 2021 Proceedings, Part II. Lecture Notes in Computer Science, vol. 12749, Springer, Cham Switzerland, pp. 162-167, International Conference on Artificial Intelligence in Education 2021, Utrecht, Netherlands, 14/06/21. https://doi.org/10.1007/978-3-030-78270-2_29

Towards automatic content analysis of rhetorical structure in Brazilian college entrance essays. / Mello, Rafael Ferreira; Fiorentino, Giuseppe; Miranda, Péricles et al.
22nd International Conference, AIED 2021 Utrecht, The Netherlands, June 14–18, 2021 Proceedings, Part II. ed. / Ido Roll; Danielle McNamara; Sergey Sosnovsky; Rose Luckin; Vania Dimitrova. Cham Switzerland: Springer, 2021. p. 162-167 (Lecture Notes in Computer Science; Vol. 12749).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Towards automatic content analysis of rhetorical structure in Brazilian college entrance essays

AU - Mello, Rafael Ferreira

AU - Fiorentino, Giuseppe

AU - Miranda, Péricles

AU - Oliveira, Hilário

AU - Raković, Mladen

AU - Gašević, Dragan

N1 - Conference code: 22nd

PY - 2021

Y1 - 2021

N2 - Essay scorers manually look for the presence of required rhetorical categories to evaluate coherence, which is a time-consuming task. Several attempts in the literature have been reported to automate the identification of rhetorical categories in essays with machine learning. However, existing machine learning algorithms are mostly trained on content features which can lead to over-fitting and hindering model generalizability. Thus, this paper proposed a set of content-independent features to identify rhetorical categories. The best performing classifier, XGBoost, achieved performance comparable to human annotation and outperformed previous models.

AB - Essay scorers manually look for the presence of required rhetorical categories to evaluate coherence, which is a time-consuming task. Several attempts in the literature have been reported to automate the identification of rhetorical categories in essays with machine learning. However, existing machine learning algorithms are mostly trained on content features which can lead to over-fitting and hindering model generalizability. Thus, this paper proposed a set of content-independent features to identify rhetorical categories. The best performing classifier, XGBoost, achieved performance comparable to human annotation and outperformed previous models.

KW - Content analytics

KW - Essay analysis

KW - Rhetoric structure

UR - https://www.scopus.com/pages/publications/85126243087

U2 - 10.1007/978-3-030-78270-2_29

DO - 10.1007/978-3-030-78270-2_29

M3 - Conference Paper

AN - SCOPUS:85126243087

SN - 9783030782696

T3 - Lecture Notes in Computer Science

SP - 162

EP - 167

BT - 22nd International Conference, AIED 2021 Utrecht, The Netherlands, June 14–18, 2021 Proceedings, Part II

A2 - Roll, Ido

A2 - McNamara, Danielle

A2 - Sosnovsky, Sergey

A2 - Luckin, Rose

A2 - Dimitrova, Vania

PB - Springer

CY - Cham Switzerland

T2 - International Conference on Artificial Intelligence in Education 2021

Y2 - 14 June 2021 through 18 June 2021

ER -

Mello RF, Fiorentino G, Miranda P, Oliveira H, Raković M , Gašević D. Towards automatic content analysis of rhetorical structure in Brazilian college entrance essays. In Roll I, McNamara D, Sosnovsky S, Luckin R, Dimitrova V, editors, 22nd International Conference, AIED 2021 Utrecht, The Netherlands, June 14–18, 2021 Proceedings, Part II. Cham Switzerland: Springer. 2021. p. 162-167. (Lecture Notes in Computer Science). doi: 10.1007/978-3-030-78270-2_29